Jul 13 19:19:32.563137 ip-10-0-137-89 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 19:19:32.563148 ip-10-0-137-89 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 19:19:32.563155 ip-10-0-137-89 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 19:19:32.563416 ip-10-0-137-89 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 19:19:42.688082 ip-10-0-137-89 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 19:19:42.688097 ip-10-0-137-89 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e628e69bab824b1bac23485e9f36ba79 --
Jul 13 19:21:42.358075 ip-10-0-137-89 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 19:21:42.706593 ip-10-0-137-89 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 19:21:42.706593 ip-10-0-137-89 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 19:21:42.706593 ip-10-0-137-89 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 19:21:42.706593 ip-10-0-137-89 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 19:21:42.706593 ip-10-0-137-89 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 19:21:42.708164 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.708086    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 19:21:42.712859 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712844    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.712859 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712860    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712864    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712867    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712870    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712874    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712878    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712882    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712885    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712888    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712892    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712895    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712898    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712900    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712904    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712907    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712910    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712912    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712915    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712917    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.712923 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712925    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712929    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712933    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712936    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712939    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712942    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712945    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712947    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712950    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712953    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712955    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712958    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712960    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712963    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712965    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712968    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712970    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712972    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712975    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712977    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.713407 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712980    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712982    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712984    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712987    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712990    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712992    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712994    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712996    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.712999    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713001    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713003    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713006    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713008    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713011    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713013    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713016    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713018    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713022    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713024    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713026    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.714102 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713029    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713031    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713034    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713036    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713039    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713041    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713043    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713046    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713048    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713051    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713054    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713056    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713058    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713061    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713063    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713065    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713068    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713070    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713073    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713076    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.714655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713078    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713080    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713082    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713085    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713087    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713089    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713093    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713095    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713097    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713100    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713102    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713104    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713107    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713109    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713111    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713114    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713116    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713119    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.715232 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.713121    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.715886 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715870    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.715886 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715884    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.715886 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715887    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715891    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715894    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715899    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715904    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715907    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715909    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715912    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715915    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715918    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715920    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715923    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715925    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715928    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715930    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715932    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715935    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715937    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715940    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.715971 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715942    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715944    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715946    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715949    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715951    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715954    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715956    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715962    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715965    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715967    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715970    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715972    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715975    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715978    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715981    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715983    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715986    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715988    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715991    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715993    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.716473 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715995    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.715998    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716000    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716002    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716005    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716007    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716010    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716012    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716014    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716018    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716020    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716022    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716025    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716027    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716030    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716033    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716036    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716038    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716041    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716043    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.716965 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716045    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716049    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716052    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716055    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716058    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716062    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716064    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716067    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716069    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716072    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716074    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716077    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716080    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716084    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716086    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716089    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716091    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716094    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716096    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.717446 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716098    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716101    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716103    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716106    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716109    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716111    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716113    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716116    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716119    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716123    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716125    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716128    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716130    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716133    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716135    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716138    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716140    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716143    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716145    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716505    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.717915 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716510    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716513    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716515    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716518    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716520    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716523    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716525    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716528    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716531    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716534    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716540    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716543    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716546    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716548    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716551    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716555    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716557    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716560    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716562    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.718421 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716565    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716568    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716570    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716572    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716575    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716577    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716580    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716582    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716585    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716587    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716590    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716592    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716594    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716597    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716599    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716601    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716604    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716606    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716609    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716612    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.718906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716616    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716618    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716621    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716623    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716625    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716628    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716630    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716633    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716635    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716638    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716640    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716643    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716645    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716648    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716650    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716653    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716655    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716658    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716660    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716664    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.719379 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716668    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716671    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716673    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716676    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716678    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716682    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716686    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716688    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716691    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716694    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716696    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716718    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716721    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716724    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716727    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716729    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716731    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716734    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716737    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.719860 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716739    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716741    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716744    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716749    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716752    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716754    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716757    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716759    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716762    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716764    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716767    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716769    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716772    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716774    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716777    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716779    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716781    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716784    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716787    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.716789    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.720301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716860    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716870    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716879    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716885    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716891    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716895    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716901    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716907    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716910    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716913    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716916    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716919    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716922    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716925    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716928    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716931    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716934    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716937    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716940    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716943    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716946    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716949    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716952    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716956    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 19:21:42.720793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716959    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716961    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716965    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716967    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716970    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716973    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716976    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716979    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716983    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716986    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716988    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716991    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716994    2562 flags.go:64] FLAG: --enable-server="true"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.716997    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717002    2562 flags.go:64] FLAG: --event-burst="100"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717005    2562 flags.go:64] FLAG: --event-qps="50"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717008    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717010    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717013    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717016    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717019    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717021    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717024    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717027    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717030    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 19:21:42.721439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717033    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717036    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717039    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717044    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717046    2562 flags.go:64] FLAG: --feature-gates=""
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717050    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717053    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717055    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717058    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717061    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717064    2562 flags.go:64] FLAG: --help="false"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717067    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-137-89.ec2.internal"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717070    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717073    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717075    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717078    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717082    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717084    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717087    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717089    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717094    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717097    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717100    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717103    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 13 19:21:42.722098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717106    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717108    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717111    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717114    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717116    2562 flags.go:64] FLAG: --lock-file=""
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717119    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717122    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717125    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717130    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717133    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717135    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717139    2562 flags.go:64] FLAG: --logging-format="text"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717142    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717145    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717148    2562 flags.go:64] FLAG: --manifest-url=""
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717151    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717155    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717157    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717161    2562 flags.go:64] FLAG: --max-pods="110"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717164    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717167    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717169    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717172    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717174    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717177    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 19:21:42.722715 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717180    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717187    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717190    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717194    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717197    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717200    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717204    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717207    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717210    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717212    2562 flags.go:64] FLAG: --port="10250"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717215    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717218    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0510d91c01c2fef26"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717221    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717224    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717226    2562 flags.go:64] FLAG: --register-node="true"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717229    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717237    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717240    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717244    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717247    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717250    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717253    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717256    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717259    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717262    2562 flags.go:64] FLAG: --runonce="false"
Jul 13 19:21:42.723356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717264    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717267    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717270    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717273    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717275    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717278    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717281    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717284    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717287    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717289    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717292    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717295    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717298    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717301    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717304    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717308    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717311    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717314    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717317    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717320    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717323    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717325    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717328    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717331    2562 flags.go:64] FLAG: --v="2"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717334    2562 flags.go:64] FLAG: --version="false"
Jul 13 19:21:42.723968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717340    2562 flags.go:64] FLAG: --vmodule=""
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717344    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.717347    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717460    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717464    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717466    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717469    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717471    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717474    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717476    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717479    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717481    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717483    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717486    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717489    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717491    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717494    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717496    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717500    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717502    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.724558 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717505    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717507    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717509    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717512    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717514    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717517    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717520    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717522    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717524    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717530    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717532    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717535    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717538    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717542    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717544    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717546    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717549    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717551    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717554    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717556    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.725050 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717558    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717561    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717563    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717565    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717567    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717570    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717572    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717574    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717577    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717580    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717584    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717586    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717589    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717591    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717594    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717596    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717599    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717601    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717604    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.725572 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717608    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717611    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717614    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717617    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717620    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717623    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717626    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717628    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717631    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717633    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717636    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717638    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717640    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717643    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717646    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717649    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717653    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717656    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717659    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.726081 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717661    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717664    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717667    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717669    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717673    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717675    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717678    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717680    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717683    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717685    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717687    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717690    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717692    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717695    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717697    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717712    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717715    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717717    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717721    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717723    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.726542 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717726    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717729    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717732    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.717735    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.718285    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.724410    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.724425    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.725085    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.725092    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725162    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725167    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725171    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725174    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725177    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725180    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.727029 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725183    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725186    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725189    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725191    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725194    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725197    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725200    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725202    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725205    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725207    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725210    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725212    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725214    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725217    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725220    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725222    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725224    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725227    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725229    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.727398 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725233    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725236    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725239    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725242    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725246    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725249    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725251    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725254    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725256    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725258    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725261    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725263    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725266    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725269    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725271    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725273    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725276    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725279    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725281    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725285    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.727909 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725289    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725292    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725295    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725299    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725302    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725304    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725306    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725309    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725312    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725314    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725316    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725319    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725321    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725323    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725326    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725328    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725331    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725334    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725337    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725339    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.728393 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725341    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725344    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725346    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725348    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725351    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725353    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725356    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725358    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725360    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725363    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725365    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725368    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725370    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725373    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725375    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725378    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725380    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725382    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725385    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725387    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.729028 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725390    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725392    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725394    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725397    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725399    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725401    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725403    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725406    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725410    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725412    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725415    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725417    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725419    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725422    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.725427    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 19:21:42.729503 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725549    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725554    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725557    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725561    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725565    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725568    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725570    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725573    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725577    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725579    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725582    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725585    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725588    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725590    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725593    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725595    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725598    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725600    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725603    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.730077 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725605    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725608    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725610    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725613    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725615    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725618    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725621    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725624    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725626    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725629    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725631    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725634    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725636    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725639    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725641    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725643    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725646    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725648    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725650    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725653    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.730966 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725655    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725657    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725660    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725662    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725665    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725667    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725670    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725673    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725676    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725678    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725681    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725683    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725686    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725688    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725690    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725692    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725695    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725697    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725712    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.731466 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725715    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725718    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725720    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725724    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725726    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725728    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725731    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725733    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725736    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725738    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725741    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725743    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725745    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725748    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725750    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725752    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725755    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725757    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725759    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725762    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.732183 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725764    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725767    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725769    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725771    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725774    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725776    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725779    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725781    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725783    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725786    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725788    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725790    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725793    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725796    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725798    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725802    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725805    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725808    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725811    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.732655 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725814    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:42.725816    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.725821    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.725919    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.727655    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.728682    2562 server.go:1019] "Starting client certificate rotation"
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.728811    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.728863    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 19:21:42.733122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.730734    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 19:21:42.733369 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.733182    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 19:21:42.741933 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.741906    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 13 19:21:42.743550 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.743536    2562 log.go:25] "Validated CRI v1 image API"
Jul 13 19:21:42.744325 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.744307    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 19:21:42.746988 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.746970    2562 fs.go:135] Filesystem UUIDs: map[36349e8c-2b63-4a83-877d-fcc2526e6c54:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d9d1378b-33ad-4e58-80dd-6c263052633f:/dev/nvme0n1p4]
Jul 13 19:21:42.747038 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.746988    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 19:21:42.752661 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.752553    2562 manager.go:216] Machine: {Timestamp:2026-07-13 19:21:42.751215617 +0000 UTC m=+0.285324049 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106100 MemoryCapacity:32812183552 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2996058e72ebeef0c5629a3df518f9 SystemUUID:ec299605-8e72-ebee-f0c5-629a3df518f9 BootID:e628e69b-ab82-4b1b-ac23-485e9f36ba79 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406093824 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:02:63:62:a7:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:02:63:62:a7:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:80:1e:ab:f7:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812183552 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 19:21:42.752661 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.752659    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 19:21:42.752786 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.752742    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 19:21:42.755072 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.755052    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 19:21:42.755192 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.755074    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-89.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 19:21:42.755645 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.755634    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 19:21:42.755677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.755647    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 19:21:42.755677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.755655    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 19:21:42.756271 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.756260    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 19:21:42.757235 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.757225    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 19:21:42.759066 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759050    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 19:21:42.759816 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759805    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 19:21:42.759928 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759919    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 19:21:42.759963 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759955    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 19:21:42.759994 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759969    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 19:21:42.759994 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759983    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 19:21:42.759994 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.759992    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 13 19:21:42.760091 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.760000    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 19:21:42.760318 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.760307    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 19:21:42.760358 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.760327    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 19:21:42.763613 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.763597    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 19:21:42.765534 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765520    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 19:21:42.765612 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765550    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 19:21:42.765694 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765682    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 19:21:42.765766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765717    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 19:21:42.765766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765725    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 19:21:42.765766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765730    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 19:21:42.765766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765738    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 19:21:42.765766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.765743    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 19:21:42.766183 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766172    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 19:21:42.766183 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766183    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 19:21:42.766242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766190    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 19:21:42.766242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766196    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 19:21:42.766242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766209    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 19:21:42.766242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766217    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 19:21:42.766761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766748    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 19:21:42.766794 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.766764    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 19:21:42.770451 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.770437    2562 server.go:1268] "Started kubelet"
Jul 13 19:21:42.770557 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.770517    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 19:21:42.770852 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.770675    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 19:21:42.773953 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.773882    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 19:21:42.774164 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.774139    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 19:21:42.774520 ip-10-0-137-89 systemd[1]: Started Kubernetes Kubelet.
Jul 13 19:21:42.774691 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.774655    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 19:21:42.774771 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.774687    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-89.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 19:21:42.775255 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.775165    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 19:21:42.776725 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.776685    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 19:21:42.777552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.777540    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 19:21:42.781066 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781050    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 19:21:42.781147 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.781133    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:42.781206 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781134    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 19:21:42.781616 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781595    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 19:21:42.781616 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781618    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 19:21:42.781812 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.780809    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-89.ec2.internal.18c1efb86b67bed9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-89.ec2.internal,UID:ip-10-0-137-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-89.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.770409177 +0000 UTC m=+0.304517610,LastTimestamp:2026-07-13 19:21:42.770409177 +0000 UTC m=+0.304517610,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-89.ec2.internal,}"
Jul 13 19:21:42.781812 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781596    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 19:21:42.781812 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781780    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 19:21:42.781812 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781788    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 19:21:42.781997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781976    2562 factory.go:55] Registering systemd factory
Jul 13 19:21:42.781997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.781990    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 13 19:21:42.782231 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.782216    2562 factory.go:153] Registering CRI-O factory
Jul 13 19:21:42.782231 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.782232    2562 factory.go:223] Registration of the crio container factory successfully
Jul 13 19:21:42.782364 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.782281    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 19:21:42.782364 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.782304    2562 factory.go:103] Registering Raw factory
Jul 13 19:21:42.782364 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.782319    2562 manager.go:1196] Started watching for new ooms in manager
Jul 13 19:21:42.782632 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.782620    2562 manager.go:349] Starting recovery of all containers
Jul 13 19:21:42.786626 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.786405    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 19:21:42.786626 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.786631    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 19:21:42.790356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.790334    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f48hq"
Jul 13 19:21:42.794720 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.794691    2562 manager.go:354] Recovery completed
Jul 13 19:21:42.800050 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.800036    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:42.802007 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.801988    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:42.802111 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802024    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:42.802111 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802038    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:42.802254 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802235    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f48hq"
Jul 13 19:21:42.802600 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802583    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 19:21:42.802652 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802600    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 19:21:42.802652 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802614    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 19:21:42.802652 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.802644    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:42.803369 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.803350    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:42.803442 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.803383    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:42.803442 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.803394    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:42.803855 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.803825    2562 policy_none.go:49] "None policy: Start"
Jul 13 19:21:42.803903 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.803861    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 19:21:42.803903 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.803874    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 19:21:42.804720 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.804687    2562 policy_none.go:47] "Start"
Jul 13 19:21:42.804720 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.804638    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-89.ec2.internal.18c1efb86d49e2da  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-89.ec2.internal,UID:ip-10-0-137-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-89.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-89.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.802006746 +0000 UTC m=+0.336115181,LastTimestamp:2026-07-13 19:21:42.802006746 +0000 UTC m=+0.336115181,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-89.ec2.internal,}"
Jul 13 19:21:42.843239 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843225    2562 manager.go:337] "Starting Device Plugin manager"
Jul 13 19:21:42.843314 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.843259    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 19:21:42.843314 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843272    2562 server.go:87] "Starting device plugin registration server"
Jul 13 19:21:42.843505 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843493    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 19:21:42.843546 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843506    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 19:21:42.843601 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843585    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 19:21:42.843666 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843646    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 19:21:42.843666 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.843656    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 19:21:42.844165 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.844144    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 19:21:42.844232 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.844197    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:42.944101 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.944076    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:42.945214 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.945189    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:42.945289 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.945231    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:42.945289 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.945243    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:42.945289 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.945269    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-89.ec2.internal"
Jul 13 19:21:42.960587 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.960543    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-89.ec2.internal"
Jul 13 19:21:42.960587 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.960565    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-89.ec2.internal\": node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:42.964579 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.964557    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 19:21:42.965899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.965881    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 19:21:42.965964 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.965931    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 19:21:42.965964 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.965955    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 19:21:42.966031 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.965996    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 19:21:42.967816 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:42.967793    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:42.984850 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:42.984823    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.066953 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.066892    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"]
Jul 13 19:21:43.067118 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.066992    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.068011 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.067994    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.068084 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.068030    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.068084 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.068043    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.069189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.069177    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.069299 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.069279    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.069351 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.069312    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.069962 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.069946    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.070034 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.069968    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.070034 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.069980    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.070108 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.070032    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.070108 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.070060    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.070108 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.070077    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.071042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.071027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.071089 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.071058    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.071719 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.071689    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.071779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.071733    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.071779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.071743    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.085655 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.085636    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.100217 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.100200    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-89.ec2.internal\" not found" node="ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.104420 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.104406    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-89.ec2.internal\" not found" node="ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.182275 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.182250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/337d027ddbda42a4cd3b3abf536a25de-config\") pod \"kube-apiserver-proxy-ip-10-0-137-89.ec2.internal\" (UID: \"337d027ddbda42a4cd3b3abf536a25de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.182366 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.182282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de3f0c058702c4bc5b538ac4c27a4cf9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal\" (UID: \"de3f0c058702c4bc5b538ac4c27a4cf9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.182366 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.182300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de3f0c058702c4bc5b538ac4c27a4cf9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal\" (UID: \"de3f0c058702c4bc5b538ac4c27a4cf9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.186392 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.186376    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.283114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.283049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/337d027ddbda42a4cd3b3abf536a25de-config\") pod \"kube-apiserver-proxy-ip-10-0-137-89.ec2.internal\" (UID: \"337d027ddbda42a4cd3b3abf536a25de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.283114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.283079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de3f0c058702c4bc5b538ac4c27a4cf9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal\" (UID: \"de3f0c058702c4bc5b538ac4c27a4cf9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.283114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.283096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de3f0c058702c4bc5b538ac4c27a4cf9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal\" (UID: \"de3f0c058702c4bc5b538ac4c27a4cf9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.283284 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.283148    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de3f0c058702c4bc5b538ac4c27a4cf9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal\" (UID: \"de3f0c058702c4bc5b538ac4c27a4cf9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.283284 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.283164    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de3f0c058702c4bc5b538ac4c27a4cf9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal\" (UID: \"de3f0c058702c4bc5b538ac4c27a4cf9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.283284 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.283173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/337d027ddbda42a4cd3b3abf536a25de-config\") pod \"kube-apiserver-proxy-ip-10-0-137-89.ec2.internal\" (UID: \"337d027ddbda42a4cd3b3abf536a25de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.287170 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.287151    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.387610 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.387575    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.401787 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.401764    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.406829 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.406811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:43.488335 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.488309    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.588933 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.588881    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.689363 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.689339    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.728646 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.728630    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 19:21:43.729195 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.728776    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 19:21:43.790049 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:43.790006    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:43.805137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.805104    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 19:16:43 +0000 UTC" deadline="2028-04-13 20:05:00.416551104 +0000 UTC"
Jul 13 19:21:43.805137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:43.805130    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15360h43m16.611424685s"
Jul 13 19:21:45.009636 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:45.009612    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:45.064059 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.064033    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:45.110288 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:45.110259    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:45.150994 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:45.150965    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337d027ddbda42a4cd3b3abf536a25de.slice/crio-8cad72acd2f48b3efc481144925310382da490a56a871037cfbabf488ed65ce1 WatchSource:0}: Error finding container 8cad72acd2f48b3efc481144925310382da490a56a871037cfbabf488ed65ce1: Status 404 returned error can't find the container with id 8cad72acd2f48b3efc481144925310382da490a56a871037cfbabf488ed65ce1
Jul 13 19:21:45.151243 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:45.151221    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3f0c058702c4bc5b538ac4c27a4cf9.slice/crio-5e391ac8d81ae9650b29298e5bc9700b1a681c55a717cc1540d67bcf4aca6d43 WatchSource:0}: Error finding container 5e391ac8d81ae9650b29298e5bc9700b1a681c55a717cc1540d67bcf4aca6d43: Status 404 returned error can't find the container with id 5e391ac8d81ae9650b29298e5bc9700b1a681c55a717cc1540d67bcf4aca6d43
Jul 13 19:21:45.155887 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.155873    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:21:45.210874 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:45.210848    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-89.ec2.internal\" not found"
Jul 13 19:21:45.216969 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.216955    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:45.270564 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.270486    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:45.300759 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.300737    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:45.316664 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.316645    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 19:21:45.317570 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.317558    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal"
Jul 13 19:21:45.330232 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.330215    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 19:21:45.879930 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.879906    2562 apiserver.go:52] "Watching apiserver"
Jul 13 19:21:45.889164 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.888992    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 19:21:45.890045 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.890022    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-6cg2s","openshift-image-registry/node-ca-n5b2j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal","openshift-multus/multus-x87dw","openshift-multus/network-metrics-daemon-lnzkp","openshift-network-operator/iptables-alerter-tw9c7","kube-system/global-pull-secret-syncer-276kq","kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2","openshift-multus/multus-additional-cni-plugins-hcpgl","openshift-network-diagnostics/network-check-target-twhgh","openshift-ovn-kubernetes/ovnkube-node-mw2d8","kube-system/konnectivity-agent-rq9wv","openshift-cluster-node-tuning-operator/tuned-hfbt2"]
Jul 13 19:21:45.891357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.891332    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:45.891464 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:45.891418    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:45.892580 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.892558    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:45.893934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.893916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.894936 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.894906    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.896036 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.896018    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:45.896127 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:45.896079    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:45.897940 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.897161    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 19:21:45.897940 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.897252    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:45.897940 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.897337    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 19:21:45.897940 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.897252    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q87nl\""
Jul 13 19:21:45.899861 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.898757    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tgw8n\""
Jul 13 19:21:45.899861 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.898865    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 19:21:45.899861 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.898981    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.899861 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.899099    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.899861 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.899166    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 19:21:45.899861 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.899292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.900341 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.900321    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:45.900439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.900359    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.900582 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.900565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zlklr\""
Jul 13 19:21:45.900770 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.900757    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.901260 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.901244    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 19:21:45.901314 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.901261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:45.903168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.902182    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 19:21:45.903168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.902521    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.903168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.902670    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2q8pt\""
Jul 13 19:21:45.903168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.902772    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.903168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.902876    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 19:21:45.903473 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.903401    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:45.904134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.903835    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.904134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.903941    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.905683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.905664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:45.906033 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.906017    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:45.906113 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:45.906072    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:45.906236 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.906221    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.906301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.906234    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 19:21:45.911052 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.911018    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 19:21:45.911268 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.911247    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 19:21:45.911370 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.911270    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.912189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.911570    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 19:21:45.912189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.911578    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7d5b6\""
Jul 13 19:21:45.912189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.911813    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.912189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.912015    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t9jw9\""
Jul 13 19:21:45.912690 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.912674    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4gpmg\""
Jul 13 19:21:45.912933 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.912918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f8cz8\""
Jul 13 19:21:45.913003 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.912971    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4scwj\""
Jul 13 19:21:45.913209 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913194    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 19:21:45.913312 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913194    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.913490 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913470    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-etc-kubernetes\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.913558 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da0e4894-383d-462d-b290-8d05c93f3ad6-konnectivity-ca\") pod \"konnectivity-agent-rq9wv\" (UID: \"da0e4894-383d-462d-b290-8d05c93f3ad6\") " pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:45.913619 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddb8134b-5daf-4c1a-b808-2244c56fdb07-tmp\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.913619 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7a75ec1-d41c-4ed3-a272-3263619f7e48-iptables-alerter-script\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:45.913711 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18aab328-ca06-489b-b325-cff43c4e62dc-multus-daemon-config\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.913711 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-systemd\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.913711 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-lib-modules\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.913851 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.913714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-tuned\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.914060 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-kubelet\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914127 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-multus-certs\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914127 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:45.914223 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914150    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk7td\" (UniqueName: \"kubernetes.io/projected/ec51aa55-8fdc-41a9-bc13-420515da094b-kube-api-access-gk7td\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:45.914274 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-sys\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.914323 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-os-release\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914323 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d59gc\" (UniqueName: \"kubernetes.io/projected/18aab328-ca06-489b-b325-cff43c4e62dc-kube-api-access-d59gc\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914414 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-var-lib-kubelet\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.914490 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-sys-fs\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.914568 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914519    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-cni-bin\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914622 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-hosts-file\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:45.914622 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-hostroot\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914721 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-host\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.914721 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-host\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:45.914721 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-system-cni-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.914859 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bt9t\" (UniqueName: \"kubernetes.io/projected/ddb8134b-5daf-4c1a-b808-2244c56fdb07-kube-api-access-2bt9t\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.914859 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb5dl\" (UniqueName: \"kubernetes.io/projected/b7a75ec1-d41c-4ed3-a272-3263619f7e48-kube-api-access-wb5dl\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:45.914859 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914803    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.914859 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sswnf\" (UniqueName: \"kubernetes.io/projected/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-kube-api-access-sswnf\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:45.915043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914880    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:45.915043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18aab328-ca06-489b-b325-cff43c4e62dc-cni-binary-copy\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.915043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-conf-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.915043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-tmp-dir\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:45.915043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.914990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-k8s-cni-cncf-io\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.915043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915023    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-socket-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.915315 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-cni-multus\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.915315 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.915412 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns8dr\" (UniqueName: \"kubernetes.io/projected/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-kube-api-access-ns8dr\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:45.915412 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-socket-dir-parent\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.915507 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysctl-conf\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.915507 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e22396dc-eaa1-48db-b917-bfea50c1c49c-kubelet-config\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:45.915623 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-serviceca\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:45.915623 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-netns\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.915623 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-modprobe-d\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.915763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915623    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysconfig\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.915763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915650    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-kubernetes\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.915763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7a75ec1-d41c-4ed3-a272-3263619f7e48-host-slash\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:45.915763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-device-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.915944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da0e4894-383d-462d-b290-8d05c93f3ad6-agent-certs\") pod \"konnectivity-agent-rq9wv\" (UID: \"da0e4894-383d-462d-b290-8d05c93f3ad6\") " pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:45.915944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-run\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.915944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjkht\" (UniqueName: \"kubernetes.io/projected/19f6594c-fc25-44da-abd0-600a4bbf9c6c-kube-api-access-qjkht\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.915944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-registration-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:45.915944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e22396dc-eaa1-48db-b917-bfea50c1c49c-dbus\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:45.916187 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915947    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-cni-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.916187 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.915975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-cnibin\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:45.916187 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.916004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysctl-d\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:45.918704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.918677    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 19:21:45.918910 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.918894    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 19:21:45.919094 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.919074    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 19:21:45.919292 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.919273    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 19:21:45.926376 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.926353    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 19:21:45.951962 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.951925    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dfxfz"
Jul 13 19:21:45.962698 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:45.962677    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dfxfz"
Jul 13 19:21:46.001461 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.001442    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 19:21:46.016811 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da0e4894-383d-462d-b290-8d05c93f3ad6-agent-certs\") pod \"konnectivity-agent-rq9wv\" (UID: \"da0e4894-383d-462d-b290-8d05c93f3ad6\") " pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:46.016927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-run\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.016927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjkht\" (UniqueName: \"kubernetes.io/projected/19f6594c-fc25-44da-abd0-600a4bbf9c6c-kube-api-access-qjkht\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.016927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-etc-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.016927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-registration-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e22396dc-eaa1-48db-b917-bfea50c1c49c-dbus\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-cni-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.016982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-cnibin\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysctl-d\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-systemd-units\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-etc-kubernetes\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da0e4894-383d-462d-b290-8d05c93f3ad6-konnectivity-ca\") pod \"konnectivity-agent-rq9wv\" (UID: \"da0e4894-383d-462d-b290-8d05c93f3ad6\") " pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddb8134b-5daf-4c1a-b808-2244c56fdb07-tmp\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.017179 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7a75ec1-d41c-4ed3-a272-3263619f7e48-iptables-alerter-script\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-slash\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovn-node-metrics-cert\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017276    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18aab328-ca06-489b-b325-cff43c4e62dc-multus-daemon-config\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-systemd\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-lib-modules\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-tuned\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-kubelet\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-cni-bin\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssbrh\" (UniqueName: \"kubernetes.io/projected/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-kube-api-access-ssbrh\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-kubelet\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-multus-certs\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk7td\" (UniqueName: \"kubernetes.io/projected/ec51aa55-8fdc-41a9-bc13-420515da094b-kube-api-access-gk7td\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:46.017683 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-sys\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-run-netns\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovnkube-config\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-os-release\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d59gc\" (UniqueName: \"kubernetes.io/projected/18aab328-ca06-489b-b325-cff43c4e62dc-kube-api-access-d59gc\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-var-lib-kubelet\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017855    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-sys-fs\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017881    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-run-ovn-kubernetes\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-run\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-cni-bin\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-cni-bin\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.017959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-hosts-file\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-hostroot\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018060    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-host\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-log-socket\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-cni-netd\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.018552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-host\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-system-cni-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bt9t\" (UniqueName: \"kubernetes.io/projected/ddb8134b-5daf-4c1a-b808-2244c56fdb07-kube-api-access-2bt9t\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18aab328-ca06-489b-b325-cff43c4e62dc-multus-daemon-config\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.018348    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-multus-certs\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018392    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-kubelet\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.018432    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:21:46.518399559 +0000 UTC m=+2.933675742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb5dl\" (UniqueName: \"kubernetes.io/projected/b7a75ec1-d41c-4ed3-a272-3263619f7e48-kube-api-access-wb5dl\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-hosts-file\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018892    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysctl-d\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.018943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-etc-kubernetes\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da0e4894-383d-462d-b290-8d05c93f3ad6-konnectivity-ca\") pod \"konnectivity-agent-rq9wv\" (UID: \"da0e4894-383d-462d-b290-8d05c93f3ad6\") " pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:46.019510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e22396dc-eaa1-48db-b917-bfea50c1c49c-dbus\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-lib-modules\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-cni-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019657    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-var-lib-kubelet\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-sys-fs\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-sys\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019747    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-registration-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-hostroot\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sswnf\" (UniqueName: \"kubernetes.io/projected/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-kube-api-access-sswnf\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18aab328-ca06-489b-b325-cff43c4e62dc-cni-binary-copy\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019953    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-conf-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.019982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb8ml\" (UniqueName: \"kubernetes.io/projected/530fff59-f369-4bf4-a159-837593b90da5-kube-api-access-xb8ml\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-node-log\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-tmp-dir\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-systemd\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-os-release\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.020194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-k8s-cni-cncf-io\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-socket-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.020187    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-cnibin\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.020235    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:21:46.520218449 +0000 UTC m=+2.935494630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7a75ec1-d41c-4ed3-a272-3263619f7e48-iptables-alerter-script\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-systemd\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-var-lib-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-cnibin\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovnkube-script-lib\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-cni-multus\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-system-cni-dir\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.021039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-env-overrides\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-tmp-dir\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-host\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns8dr\" (UniqueName: \"kubernetes.io/projected/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-kube-api-access-ns8dr\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-system-cni-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-socket-dir-parent\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysctl-conf\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-conf-dir\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020672    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e22396dc-eaa1-48db-b917-bfea50c1c49c-kubelet-config\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e22396dc-eaa1-48db-b917-bfea50c1c49c-kubelet-config\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020714    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-serviceca\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-var-lib-cni-multus\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020744    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-netns\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-modprobe-d\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-socket-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysconfig\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.021647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-os-release\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-ovn\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-kubernetes\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18aab328-ca06-489b-b325-cff43c4e62dc-cni-binary-copy\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7a75ec1-d41c-4ed3-a272-3263619f7e48-host-slash\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-k8s-cni-cncf-io\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-device-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.020997    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-host\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-multus-socket-dir-parent\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7a75ec1-d41c-4ed3-a272-3263619f7e48-host-slash\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18aab328-ca06-489b-b325-cff43c4e62dc-host-run-netns\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-modprobe-d\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021131    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysconfig\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/19f6594c-fc25-44da-abd0-600a4bbf9c6c-device-dir\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddb8134b-5daf-4c1a-b808-2244c56fdb07-tmp\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-sysctl-conf\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-kubernetes\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddb8134b-5daf-4c1a-b808-2244c56fdb07-etc-tuned\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.022463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-serviceca\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.023294 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.021799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da0e4894-383d-462d-b290-8d05c93f3ad6-agent-certs\") pod \"konnectivity-agent-rq9wv\" (UID: \"da0e4894-383d-462d-b290-8d05c93f3ad6\") " pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:46.032344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.032255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns8dr\" (UniqueName: \"kubernetes.io/projected/8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071-kube-api-access-ns8dr\") pod \"node-ca-n5b2j\" (UID: \"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071\") " pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.032793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.032766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d59gc\" (UniqueName: \"kubernetes.io/projected/18aab328-ca06-489b-b325-cff43c4e62dc-kube-api-access-d59gc\") pod \"multus-x87dw\" (UID: \"18aab328-ca06-489b-b325-cff43c4e62dc\") " pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.033436 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.033414    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk7td\" (UniqueName: \"kubernetes.io/projected/ec51aa55-8fdc-41a9-bc13-420515da094b-kube-api-access-gk7td\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:46.033987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.033958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjkht\" (UniqueName: \"kubernetes.io/projected/19f6594c-fc25-44da-abd0-600a4bbf9c6c-kube-api-access-qjkht\") pod \"aws-ebs-csi-driver-node-8qfx2\" (UID: \"19f6594c-fc25-44da-abd0-600a4bbf9c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.034636 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.034591    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bt9t\" (UniqueName: \"kubernetes.io/projected/ddb8134b-5daf-4c1a-b808-2244c56fdb07-kube-api-access-2bt9t\") pod \"tuned-hfbt2\" (UID: \"ddb8134b-5daf-4c1a-b808-2244c56fdb07\") " pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.034945 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.034929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb5dl\" (UniqueName: \"kubernetes.io/projected/b7a75ec1-d41c-4ed3-a272-3263619f7e48-kube-api-access-wb5dl\") pod \"iptables-alerter-tw9c7\" (UID: \"b7a75ec1-d41c-4ed3-a272-3263619f7e48\") " pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.037237 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.037205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sswnf\" (UniqueName: \"kubernetes.io/projected/94e22fbe-3c4c-4c65-946a-a6a2be93b2b7-kube-api-access-sswnf\") pod \"node-resolver-6cg2s\" (UID: \"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7\") " pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.089428 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.089373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal" event={"ID":"337d027ddbda42a4cd3b3abf536a25de","Type":"ContainerStarted","Data":"8cad72acd2f48b3efc481144925310382da490a56a871037cfbabf488ed65ce1"}
Jul 13 19:21:46.090549 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.090507    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal" event={"ID":"de3f0c058702c4bc5b538ac4c27a4cf9","Type":"ContainerStarted","Data":"5e391ac8d81ae9650b29298e5bc9700b1a681c55a717cc1540d67bcf4aca6d43"}
Jul 13 19:21:46.122273 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-etc-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.122420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-systemd-units\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-slash\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-etc-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-systemd-units\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovn-node-metrics-cert\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122475    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-slash\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-kubelet\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-cni-bin\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssbrh\" (UniqueName: \"kubernetes.io/projected/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-kube-api-access-ssbrh\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-cni-bin\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122592    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-run-netns\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-kubelet\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovnkube-config\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-run-ovn-kubernetes\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-log-socket\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.122734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-cni-netd\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-run-netns\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-run-ovn-kubernetes\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-host-cni-netd\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-log-socket\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.122962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb8ml\" (UniqueName: \"kubernetes.io/projected/530fff59-f369-4bf4-a159-837593b90da5-kube-api-access-xb8ml\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-node-log\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-cnibin\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-systemd\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-var-lib-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovnkube-script-lib\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123216    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-cnibin\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-system-cni-dir\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-node-log\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.123464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-env-overrides\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-os-release\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovnkube-config\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-ovn\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-systemd\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.123313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-var-lib-openvswitch\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.124438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-run-ovn\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-os-release\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.124657 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124487    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/530fff59-f369-4bf4-a159-837593b90da5-system-cni-dir\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.124657 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovnkube-script-lib\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.124753 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124723    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/530fff59-f369-4bf4-a159-837593b90da5-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.124954 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.124929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-env-overrides\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.128930 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.127896    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-ovn-node-metrics-cert\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.134064 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.134011    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:46.134064 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.134035    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:46.134064 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.134045    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:46.134232 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.134106    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:46.634094488 +0000 UTC m=+3.049370661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:46.141242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.141202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb8ml\" (UniqueName: \"kubernetes.io/projected/530fff59-f369-4bf4-a159-837593b90da5-kube-api-access-xb8ml\") pod \"multus-additional-cni-plugins-hcpgl\" (UID: \"530fff59-f369-4bf4-a159-837593b90da5\") " pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.148041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.148021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssbrh\" (UniqueName: \"kubernetes.io/projected/b87b74d7-a7cd-4c22-9bd1-e1a825d52008-kube-api-access-ssbrh\") pod \"ovnkube-node-mw2d8\" (UID: \"b87b74d7-a7cd-4c22-9bd1-e1a825d52008\") " pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.213399 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.213373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:21:46.224591 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.224568    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hfbt2"
Jul 13 19:21:46.236107 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.236087    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x87dw"
Jul 13 19:21:46.243171 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.243150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tw9c7"
Jul 13 19:21:46.244113 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.244088    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:46.250298 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.250279    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2"
Jul 13 19:21:46.259126 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.259101    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6cg2s"
Jul 13 19:21:46.265604 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.265589    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n5b2j"
Jul 13 19:21:46.273545 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.273500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcpgl"
Jul 13 19:21:46.279122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.279102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:21:46.526654 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.526578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:46.526654 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.526636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:46.526863 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.526739    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:46.526863 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.526815    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:21:47.526794969 +0000 UTC m=+3.942071143 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:46.526863 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.526740    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:46.527024 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.526890    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:21:47.526871355 +0000 UTC m=+3.942147528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:46.728283 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.728249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:46.728428 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.728413    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:46.728478 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.728439    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:46.728478 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.728452    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:46.728561 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:46.728518    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:47.728499076 +0000 UTC m=+4.143775257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:46.965038 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.964958    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:16:45 +0000 UTC" deadline="2027-12-13 12:22:27.826667153 +0000 UTC"
Jul 13 19:21:46.965038 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:46.964987    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12425h0m40.861683816s"
Jul 13 19:21:47.073283 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:47.073068    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87b74d7_a7cd_4c22_9bd1_e1a825d52008.slice/crio-b2a90504cede1170298ec773e56b2149458e4da608bd30ad093f6fa0fba1b388 WatchSource:0}: Error finding container b2a90504cede1170298ec773e56b2149458e4da608bd30ad093f6fa0fba1b388: Status 404 returned error can't find the container with id b2a90504cede1170298ec773e56b2149458e4da608bd30ad093f6fa0fba1b388
Jul 13 19:21:47.080024 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:47.079991    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f6594c_fc25_44da_abd0_600a4bbf9c6c.slice/crio-0f294f46ae9c919cfa2e4155f0cf55d694041bcd467ad7d0274c5b181114c831 WatchSource:0}: Error finding container 0f294f46ae9c919cfa2e4155f0cf55d694041bcd467ad7d0274c5b181114c831: Status 404 returned error can't find the container with id 0f294f46ae9c919cfa2e4155f0cf55d694041bcd467ad7d0274c5b181114c831
Jul 13 19:21:47.080245 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:47.080226    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530fff59_f369_4bf4_a159_837593b90da5.slice/crio-c2a121bd0f369a7e20643123cce5f6026e874fc4374e51c856183b589ea223ea WatchSource:0}: Error finding container c2a121bd0f369a7e20643123cce5f6026e874fc4374e51c856183b589ea223ea: Status 404 returned error can't find the container with id c2a121bd0f369a7e20643123cce5f6026e874fc4374e51c856183b589ea223ea
Jul 13 19:21:47.081947 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:47.081923    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb8134b_5daf_4c1a_b808_2244c56fdb07.slice/crio-36c991a96aa2e090dd38ebf333af0789816377ae43fbe8bf423e1dcb2096725d WatchSource:0}: Error finding container 36c991a96aa2e090dd38ebf333af0789816377ae43fbe8bf423e1dcb2096725d: Status 404 returned error can't find the container with id 36c991a96aa2e090dd38ebf333af0789816377ae43fbe8bf423e1dcb2096725d
Jul 13 19:21:47.083504 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:47.083373    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0e4894_383d_462d_b290_8d05c93f3ad6.slice/crio-136aa233abc70c8e37d59d5b7c7f2bc2e91c997a3d82ecd0df9897e910d562fe WatchSource:0}: Error finding container 136aa233abc70c8e37d59d5b7c7f2bc2e91c997a3d82ecd0df9897e910d562fe: Status 404 returned error can't find the container with id 136aa233abc70c8e37d59d5b7c7f2bc2e91c997a3d82ecd0df9897e910d562fe
Jul 13 19:21:47.084373 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:21:47.084337    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18aab328_ca06_489b_b325_cff43c4e62dc.slice/crio-317c23ca650b3526117dbe6e83ea42f3fe05d3b6e096e7b0f485970576a6ddc4 WatchSource:0}: Error finding container 317c23ca650b3526117dbe6e83ea42f3fe05d3b6e096e7b0f485970576a6ddc4: Status 404 returned error can't find the container with id 317c23ca650b3526117dbe6e83ea42f3fe05d3b6e096e7b0f485970576a6ddc4
Jul 13 19:21:47.085222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.085034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:47.085222 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.085177    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:47.093128 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.093059    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2" event={"ID":"19f6594c-fc25-44da-abd0-600a4bbf9c6c","Type":"ContainerStarted","Data":"0f294f46ae9c919cfa2e4155f0cf55d694041bcd467ad7d0274c5b181114c831"}
Jul 13 19:21:47.094273 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.094243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6cg2s" event={"ID":"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7","Type":"ContainerStarted","Data":"1609c40c095278893e954e915e1481c830046cd195a08eb5542c01ac25df4dd8"}
Jul 13 19:21:47.095414 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.095392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"b2a90504cede1170298ec773e56b2149458e4da608bd30ad093f6fa0fba1b388"}
Jul 13 19:21:47.096446 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.096425    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rq9wv" event={"ID":"da0e4894-383d-462d-b290-8d05c93f3ad6","Type":"ContainerStarted","Data":"136aa233abc70c8e37d59d5b7c7f2bc2e91c997a3d82ecd0df9897e910d562fe"}
Jul 13 19:21:47.097745 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.097716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tw9c7" event={"ID":"b7a75ec1-d41c-4ed3-a272-3263619f7e48","Type":"ContainerStarted","Data":"f52a541c21e1017435cf624eb2d6448d25e5e16ab65cce12c40effa33045f4f0"}
Jul 13 19:21:47.098946 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.098906    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n5b2j" event={"ID":"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071","Type":"ContainerStarted","Data":"f52ecfa7fb53c745456105c9a041b06bd029872c43fd437d60958eeb9885c5ce"}
Jul 13 19:21:47.100291 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.100258    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x87dw" event={"ID":"18aab328-ca06-489b-b325-cff43c4e62dc","Type":"ContainerStarted","Data":"317c23ca650b3526117dbe6e83ea42f3fe05d3b6e096e7b0f485970576a6ddc4"}
Jul 13 19:21:47.102159 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.102115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hfbt2" event={"ID":"ddb8134b-5daf-4c1a-b808-2244c56fdb07","Type":"ContainerStarted","Data":"36c991a96aa2e090dd38ebf333af0789816377ae43fbe8bf423e1dcb2096725d"}
Jul 13 19:21:47.103388 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.103365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerStarted","Data":"c2a121bd0f369a7e20643123cce5f6026e874fc4374e51c856183b589ea223ea"}
Jul 13 19:21:47.533419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.533339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:47.533419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.533378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:47.533608 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.533488    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:47.533608 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.533488    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:47.533608 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.533565    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:21:49.533529375 +0000 UTC m=+5.948805535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:47.533608 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.533579    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:21:49.533572538 +0000 UTC m=+5.948848709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:47.734186 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.734150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:47.734355 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.734320    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:47.734355 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.734343    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:47.734355 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.734355    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:47.734509 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:47.734414    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:49.734394932 +0000 UTC m=+6.149671115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:47.965983 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.965876    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:16:45 +0000 UTC" deadline="2028-01-05 15:55:21.432246639 +0000 UTC"
Jul 13 19:21:47.965983 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:47.965933    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12980h33m33.466318141s"
Jul 13 19:21:48.089024 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:48.088197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:48.089024 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:48.088334    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:48.089024 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:48.088816    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:48.089024 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:48.088957    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:48.112661 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:48.112629    2562 generic.go:356] "Generic (PLEG): container finished" podID="de3f0c058702c4bc5b538ac4c27a4cf9" containerID="116f3e840bc063d6b5f0b4c1fda23a7bae452098895d65d4eb1c7416f1169ccb" exitCode=0
Jul 13 19:21:48.112850 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:48.112821    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal" event={"ID":"de3f0c058702c4bc5b538ac4c27a4cf9","Type":"ContainerDied","Data":"116f3e840bc063d6b5f0b4c1fda23a7bae452098895d65d4eb1c7416f1169ccb"}
Jul 13 19:21:48.114860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:48.114835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal" event={"ID":"337d027ddbda42a4cd3b3abf536a25de","Type":"ContainerStarted","Data":"28679a777b989d11eb261545cda1d13808666ff0a9fbcf9fbd7e220283f874da"}
Jul 13 19:21:48.148312 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:48.148261    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-89.ec2.internal" podStartSLOduration=3.148238534 podStartE2EDuration="3.148238534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:21:48.14779813 +0000 UTC m=+4.563074316" watchObservedRunningTime="2026-07-13 19:21:48.148238534 +0000 UTC m=+4.563514717"
Jul 13 19:21:49.085471 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:49.085434    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:49.085973 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.085606    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:49.139210 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:49.139165    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal" event={"ID":"de3f0c058702c4bc5b538ac4c27a4cf9","Type":"ContainerStarted","Data":"074bbebb93b253f99025496337443a168e456b52cc51a25d08d5da27e235c51b"}
Jul 13 19:21:49.550433 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:49.549643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:49.550433 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:49.549705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:49.550433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.549840    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:49.550433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.549905    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:21:53.549887205 +0000 UTC m=+9.965163379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:49.550433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.549974    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:49.550433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.550006    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:21:53.54999456 +0000 UTC m=+9.965270729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:49.751371 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:49.751250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:49.751600 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.751459    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:49.751600 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.751487    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:49.751600 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.751500    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:49.751600 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:49.751584    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:53.751565201 +0000 UTC m=+10.166841384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:50.085663 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:50.085631    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:50.086157 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:50.085762    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:50.088317 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:50.088291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:50.088423 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:50.088402    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:51.085866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:51.085830    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:51.086328 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:51.085987    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:52.088215 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:52.087780    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:52.088215 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:52.087915    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:52.088215 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:52.087784    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:52.088215 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:52.088058    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:53.085176 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:53.085129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:53.085367 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.085272    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:53.587095 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:53.587057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:53.587470 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:53.587121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:53.587470 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.587249    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:53.587470 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.587294    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:22:01.587280336 +0000 UTC m=+18.002556500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:53.587601 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.587576    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:53.587632 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.587609    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:01.587599251 +0000 UTC m=+18.002875411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:53.788681 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:53.788645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:53.788850 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.788829    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:53.788913 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.788851    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:53.788913 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.788864    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:53.789002 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:53.788917    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:01.788898353 +0000 UTC m=+18.204174528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:54.086300 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:54.086207    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:54.086455 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:54.086213    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:54.086455 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:54.086324    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:54.086455 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:54.086391    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:55.085967 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:55.085934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:55.086509 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:55.086080    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:56.087504 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:56.087435    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:56.087964 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:56.087544    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:56.087964 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:56.087945    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:56.088077 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:56.088034    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:57.085526 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:57.085497    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:57.085700 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:57.085653    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:21:58.084979 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:58.084934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:21:58.085343 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:58.085065    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:21:58.085343 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:58.085111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:21:58.085343 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:58.085181    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:21:59.085777 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:21:59.085737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:21:59.086178 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:21:59.085867    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:00.085193 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:00.085161    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:00.085369 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:00.085279    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:00.085435 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:00.085372    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:00.085502 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:00.085481    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:01.085498 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:01.085460    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:01.085975 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.085614    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:01.644207 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:01.644173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:01.644400 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:01.644218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:01.644400 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.644334    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:01.644400 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.644337    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:01.644400 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.644398    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:22:17.644380091 +0000 UTC m=+34.059656258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:01.644571 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.644413    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:17.64440556 +0000 UTC m=+34.059681720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:01.845336 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:01.845305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:01.845517 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.845504    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:22:01.845585 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.845528    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:22:01.845585 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.845554    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:01.845691 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:01.845615    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:17.845599974 +0000 UTC m=+34.260876141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:02.085848 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:02.085756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:02.086270 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:02.085897    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:02.086270 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:02.085949    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:02.086270 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:02.086066    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:03.085835 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:03.085799    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:03.085986 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:03.085944    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:04.086160 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:04.086127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:04.086572 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:04.086231    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:04.086572 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:04.086322    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:04.086572 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:04.086421    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:05.085303 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:05.085273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:05.085465 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:05.085416    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:06.085512 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:06.085477    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:06.086164 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:06.085629    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:06.086164 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:06.085686    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:06.086164 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:06.085808    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:07.085592 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.085199    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:07.086089 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:07.085677    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:07.179454 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.179423    2562 generic.go:356] "Generic (PLEG): container finished" podID="530fff59-f369-4bf4-a159-837593b90da5" containerID="61335d3a0fd9523e022934a2f9d1c84e9b6fb1275ada1e1afdaa27bcb29e9673" exitCode=0
Jul 13 19:22:07.179621 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.179508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerDied","Data":"61335d3a0fd9523e022934a2f9d1c84e9b6fb1275ada1e1afdaa27bcb29e9673"}
Jul 13 19:22:07.181196 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.181161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2" event={"ID":"19f6594c-fc25-44da-abd0-600a4bbf9c6c","Type":"ContainerStarted","Data":"0f42042000065b8a18a13c91f9bb8be84ca7b153ee63ea274cc2fe33f02af017"}
Jul 13 19:22:07.182698 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.182678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6cg2s" event={"ID":"94e22fbe-3c4c-4c65-946a-a6a2be93b2b7","Type":"ContainerStarted","Data":"7bd1b5d3c83f6bcbc6a92c5129e3898c005851413be0eff99fcd7fb1d025b29f"}
Jul 13 19:22:07.185891 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.185870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"37b4cfd2ae9d8267305186ced1e053f03bf58207012c8c92874ed7c97d425fe1"}
Jul 13 19:22:07.185976 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.185934    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"7389d1b04b4db0933b905a0b711a90720ff7be05a31bbe6284aac484656ec732"}
Jul 13 19:22:07.186039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.185977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"149e9a60a23a288e3cb8dac11604128b2bcc0b9b66f8318bff5b50b2c283a442"}
Jul 13 19:22:07.186039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.186010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"c29d04925b9f72f634a85f19de627c930df1e0a294d240fbe9723c8b41262bf9"}
Jul 13 19:22:07.186039 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.186037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"1721241e1250b9a996b356c3a967516b55dc4daafa63ad5a96c9e4798af82e52"}
Jul 13 19:22:07.186158 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.186070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"7abd2f31f3b7c083e9ce2c1ce255d1038b015dd0f7a423957b5fb4fcc14045a0"}
Jul 13 19:22:07.187272 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.187254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rq9wv" event={"ID":"da0e4894-383d-462d-b290-8d05c93f3ad6","Type":"ContainerStarted","Data":"95a5c1033aff3f32359d8157d3ba04d22525d2c64aceae3ee55ce7b1ef5dadc7"}
Jul 13 19:22:07.188687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.188663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n5b2j" event={"ID":"8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071","Type":"ContainerStarted","Data":"ef52006bb217f3f8e8cd4717e8453ded161a38fc90712778e95d0afa9ceb48ce"}
Jul 13 19:22:07.190236 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.190214    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x87dw" event={"ID":"18aab328-ca06-489b-b325-cff43c4e62dc","Type":"ContainerStarted","Data":"4aad16707ceb3a1d8ee8bb5a8475c42d4abbe4a43f878cdd9362fe404bfd3ad2"}
Jul 13 19:22:07.191572 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.191550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hfbt2" event={"ID":"ddb8134b-5daf-4c1a-b808-2244c56fdb07","Type":"ContainerStarted","Data":"37f2fff9650f58bcbf08518e64c9c75d3800ff5261c601e95ea5fdae85591e74"}
Jul 13 19:22:07.217784 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.217719    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-89.ec2.internal" podStartSLOduration=22.217687001 podStartE2EDuration="22.217687001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:21:49.159583279 +0000 UTC m=+5.574859463" watchObservedRunningTime="2026-07-13 19:22:07.217687001 +0000 UTC m=+23.632963162"
Jul 13 19:22:07.244159 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.244119    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6cg2s" podStartSLOduration=3.869341719 podStartE2EDuration="23.244109601s" totalImagesPullingTime="19.374767882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:07.244059377 +0000 UTC m=+23.659335558" watchObservedRunningTime="2026-07-13 19:22:07.244109601 +0000 UTC m=+23.659385783"
Jul 13 19:22:07.270145 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.270112    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n5b2j" podStartSLOduration=11.866892673 podStartE2EDuration="23.270101574s" totalImagesPullingTime="11.403208901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:07.270043304 +0000 UTC m=+23.685319486" watchObservedRunningTime="2026-07-13 19:22:07.270101574 +0000 UTC m=+23.685377756"
Jul 13 19:22:07.347132 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.347085    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rq9wv" podStartSLOduration=3.981742293 podStartE2EDuration="23.347070584s" totalImagesPullingTime="19.365328291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:07.34686463 +0000 UTC m=+23.762140809" watchObservedRunningTime="2026-07-13 19:22:07.347070584 +0000 UTC m=+23.762346765"
Jul 13 19:22:07.347332 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.347166    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hfbt2" podStartSLOduration=3.961637674 podStartE2EDuration="23.347158791s" totalImagesPullingTime="19.385521117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:07.316517016 +0000 UTC m=+23.731793201" watchObservedRunningTime="2026-07-13 19:22:07.347158791 +0000 UTC m=+23.762434973"
Jul 13 19:22:07.371372 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:07.371331    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x87dw" podStartSLOduration=3.938947686 podStartE2EDuration="23.371317778s" totalImagesPullingTime="19.432370092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:07.37064039 +0000 UTC m=+23.785916574" watchObservedRunningTime="2026-07-13 19:22:07.371317778 +0000 UTC m=+23.786593957"
Jul 13 19:22:08.085413 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.085383    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:08.086638 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.085383    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:08.086638 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:08.085516    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:08.086638 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:08.085606    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:08.195911 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.195815    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tw9c7" event={"ID":"b7a75ec1-d41c-4ed3-a272-3263619f7e48","Type":"ContainerStarted","Data":"83a683aeb6a589c0da4ea1f93fa4f1ca634ae98a2336d7b36f8a379a7209e550"}
Jul 13 19:22:08.213422 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.213382    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tw9c7" podStartSLOduration=4.839057285 podStartE2EDuration="24.213367563s" totalImagesPullingTime="19.374310278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:08.21279019 +0000 UTC m=+24.628066369" watchObservedRunningTime="2026-07-13 19:22:08.213367563 +0000 UTC m=+24.628643745"
Jul 13 19:22:08.432941 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.432914    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 19:22:08.665811 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.665726    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:22:08.666376 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.666352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:22:08.975974 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.975849    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T19:22:08.432937557Z","UUID":"4dad81fe-1482-4926-a5ba-a92a5cb866f1","Handler":null,"Name":"","Endpoint":""}
Jul 13 19:22:08.977413 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.977396    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 19:22:08.977548 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:08.977433    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 19:22:09.085888 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:09.085858    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:09.086060 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:09.085981    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:09.200261 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:09.200231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2" event={"ID":"19f6594c-fc25-44da-abd0-600a4bbf9c6c","Type":"ContainerStarted","Data":"fdb96fc894aacf9ca2876f1e7890c583ad518b7318c80bd2fbc18e8afd023d76"}
Jul 13 19:22:09.200645 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:09.200562    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:22:09.201154 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:09.201132    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rq9wv"
Jul 13 19:22:10.085866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:10.085642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:10.086054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:10.085720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:10.086054 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:10.085968    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:10.086168 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:10.086045    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:10.206310 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:10.206269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"8e429897108d13ae273b67c188e305ab71fb6f373728c2b2aee1e0196f509c94"}
Jul 13 19:22:11.085874 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:11.085835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:11.086086 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:11.085959    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:12.085415 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.085124    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:12.086046 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.085217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:12.086046 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:12.085474    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:12.086046 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:12.085522    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:12.212450 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.212422    2562 generic.go:356] "Generic (PLEG): container finished" podID="530fff59-f369-4bf4-a159-837593b90da5" containerID="c16ecb14f93f0007d07eafa7ae0e03d90716f1f7f31baea074935d75a113c8e6" exitCode=0
Jul 13 19:22:12.212616 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.212505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerDied","Data":"c16ecb14f93f0007d07eafa7ae0e03d90716f1f7f31baea074935d75a113c8e6"}
Jul 13 19:22:12.214854 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.214828    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2" event={"ID":"19f6594c-fc25-44da-abd0-600a4bbf9c6c","Type":"ContainerStarted","Data":"47b860813730236238fd34dcce3c87798803463b45d7bf6156589fc30fbdfe6a"}
Jul 13 19:22:12.218550 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.218497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" event={"ID":"b87b74d7-a7cd-4c22-9bd1-e1a825d52008","Type":"ContainerStarted","Data":"5314121d7dca5dbfcada197c4043aa263b79f8cd098e3f12b7d6bb55d5569101"}
Jul 13 19:22:12.218782 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.218754    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:22:12.218885 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.218824    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:22:12.218885 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.218875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:22:12.234500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.234479    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:22:12.234790 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.234774    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:22:12.285031 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:12.284984    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8" podStartSLOduration=8.830167657 podStartE2EDuration="28.284968328s" totalImagesPullingTime="19.454800671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:12.282196501 +0000 UTC m=+28.697472680" watchObservedRunningTime="2026-07-13 19:22:12.284968328 +0000 UTC m=+28.700244511"
Jul 13 19:22:13.085864 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.085839    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:13.086301 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:13.085951    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:13.462120 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.462056    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8qfx2" podStartSLOduration=5.454386727 podStartE2EDuration="29.462034822s" totalImagesPullingTime="24.007648095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:12.307290366 +0000 UTC m=+28.722566548" watchObservedRunningTime="2026-07-13 19:22:13.462034822 +0000 UTC m=+29.877311005"
Jul 13 19:22:13.462777 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.462759    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-276kq"]
Jul 13 19:22:13.462877 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.462846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:13.462961 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:13.462935    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:13.466135 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.466073    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lnzkp"]
Jul 13 19:22:13.466908 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.466879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:13.467082 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:13.467059    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:13.471762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.471739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-twhgh"]
Jul 13 19:22:13.472115 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:13.471890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:13.472115 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:13.472049    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:14.225379 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:14.225166    2562 generic.go:356] "Generic (PLEG): container finished" podID="530fff59-f369-4bf4-a159-837593b90da5" containerID="9c47e9b0d967d1af432f76c16843ee3674c0cb23f17a75f7fab52d5a46ad0556" exitCode=0
Jul 13 19:22:14.225817 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:14.225255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerDied","Data":"9c47e9b0d967d1af432f76c16843ee3674c0cb23f17a75f7fab52d5a46ad0556"}
Jul 13 19:22:15.085910 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:15.085877    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:15.086092 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:15.086019    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:15.086092 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:15.086040    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:15.086211 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:15.086102    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:15.086211 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:15.086116    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:15.086211 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:15.086195    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:16.232318 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:16.232279    2562 generic.go:356] "Generic (PLEG): container finished" podID="530fff59-f369-4bf4-a159-837593b90da5" containerID="e7a01319510dcc0730949f919e12e7c5db6381ce24c25b403dbe73bc7a68dd01" exitCode=0
Jul 13 19:22:16.232855 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:16.232352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerDied","Data":"e7a01319510dcc0730949f919e12e7c5db6381ce24c25b403dbe73bc7a68dd01"}
Jul 13 19:22:17.085730 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:17.085701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:17.085910 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:17.085701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:17.085910 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.085811    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:17.085910 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:17.085701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:17.085910 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.085865    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:17.086104 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.085946    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:17.666192 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:17.666075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:17.666192 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:17.666148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:17.666794 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.666198    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:17.666794 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.666244    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:17.666794 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.666275    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret podName:e22396dc-eaa1-48db-b917-bfea50c1c49c nodeName:}" failed. No retries permitted until 2026-07-13 19:22:49.666252961 +0000 UTC m=+66.081529124 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret") pod "global-pull-secret-syncer-276kq" (UID: "e22396dc-eaa1-48db-b917-bfea50c1c49c") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:17.666794 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.666295    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs podName:ec51aa55-8fdc-41a9-bc13-420515da094b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:49.666282568 +0000 UTC m=+66.081558731 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs") pod "network-metrics-daemon-lnzkp" (UID: "ec51aa55-8fdc-41a9-bc13-420515da094b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:17.867970 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:17.867884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:17.868124 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.868011    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:22:17.868124 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.868027    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:22:17.868124 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.868036    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4c5jl for pod openshift-network-diagnostics/network-check-target-twhgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:17.868124 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:17.868090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl podName:417eafcd-0358-421d-8d0e-f6e5960c0352 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:49.868073817 +0000 UTC m=+66.283349998 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4c5jl" (UniqueName: "kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl") pod "network-check-target-twhgh" (UID: "417eafcd-0358-421d-8d0e-f6e5960c0352") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:19.085981 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.085941    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:19.086357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.085944    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:19.086357 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.086047    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-276kq" podUID="e22396dc-eaa1-48db-b917-bfea50c1c49c"
Jul 13 19:22:19.086357 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.086144    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twhgh" podUID="417eafcd-0358-421d-8d0e-f6e5960c0352"
Jul 13 19:22:19.086357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.085943    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:19.086357 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.086260    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnzkp" podUID="ec51aa55-8fdc-41a9-bc13-420515da094b"
Jul 13 19:22:19.384411 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.384341    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-89.ec2.internal" event="NodeReady"
Jul 13 19:22:19.384576 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.384512    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 19:22:19.443635 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.443604    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-llqzj"]
Jul 13 19:22:19.445875 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.445853    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5499cbfbf-zf8bj"]
Jul 13 19:22:19.446083 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.446054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.450006 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.449982    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 19:22:19.450220 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.449989    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mf9hk\""
Jul 13 19:22:19.450317 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.450032    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 19:22:19.450629 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.450608    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l5gft"]
Jul 13 19:22:19.450755 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.450738    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.452257 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.452236    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.454131 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.454101    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 19:22:19.454246 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.454150    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 19:22:19.454246 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.454150    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 19:22:19.455074 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.455056    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-25765\""
Jul 13 19:22:19.455416 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.455397    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 19:22:19.455720 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.455700    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 19:22:19.456077 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.456055    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 19:22:19.456218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.456171    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bvq9g\""
Jul 13 19:22:19.463172 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.463144    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l5gft"]
Jul 13 19:22:19.463279 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.463149    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 19:22:19.463738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.463718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-llqzj"]
Jul 13 19:22:19.467144 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.467126    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5499cbfbf-zf8bj"]
Jul 13 19:22:19.579869 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.579840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-tmp-dir\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.580041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.579883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4ea392e-7d04-4c8a-bc09-ba81c768126b-cert\") pod \"ingress-canary-l5gft\" (UID: \"e4ea392e-7d04-4c8a-bc09-ba81c768126b\") " pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.580041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.579907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-config-volume\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.580041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.579929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.580041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.579956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz7k4\" (UniqueName: \"kubernetes.io/projected/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-kube-api-access-tz7k4\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.580041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580019    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87973b43-d163-4c3d-975d-bf54b9ef4a55-ca-trust-extracted\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580041 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnggh\" (UniqueName: \"kubernetes.io/projected/e4ea392e-7d04-4c8a-bc09-ba81c768126b-kube-api-access-nnggh\") pod \"ingress-canary-l5gft\" (UID: \"e4ea392e-7d04-4c8a-bc09-ba81c768126b\") " pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-installation-pull-secrets\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-bound-sa-token\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-image-registry-private-configuration\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-trusted-ca\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-certificates\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.580280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.580177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6c5s\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-kube-api-access-n6c5s\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.680860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.680775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-config-volume\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.680860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.680825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.680860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.680858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz7k4\" (UniqueName: \"kubernetes.io/projected/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-kube-api-access-tz7k4\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.681082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.680915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87973b43-d163-4c3d-975d-bf54b9ef4a55-ca-trust-extracted\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681082 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.681012    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:19.681082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnggh\" (UniqueName: \"kubernetes.io/projected/e4ea392e-7d04-4c8a-bc09-ba81c768126b-kube-api-access-nnggh\") pod \"ingress-canary-l5gft\" (UID: \"e4ea392e-7d04-4c8a-bc09-ba81c768126b\") " pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.681082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-installation-pull-secrets\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681082 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.681083    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls podName:ab74d8e7-399e-431b-8fbe-d57c1c8a9a59 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.181062384 +0000 UTC m=+36.596338564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls") pod "dns-default-llqzj" (UID: "ab74d8e7-399e-431b-8fbe-d57c1c8a9a59") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-bound-sa-token\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-image-registry-private-configuration\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-trusted-ca\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-certificates\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6c5s\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-kube-api-access-n6c5s\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87973b43-d163-4c3d-975d-bf54b9ef4a55-ca-trust-extracted\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.681328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-tmp-dir\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.681675 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4ea392e-7d04-4c8a-bc09-ba81c768126b-cert\") pod \"ingress-canary-l5gft\" (UID: \"e4ea392e-7d04-4c8a-bc09-ba81c768126b\") " pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.681675 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-config-volume\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.681675 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.681501    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:19.681675 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.681517    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5499cbfbf-zf8bj: secret "image-registry-tls" not found
Jul 13 19:22:19.681675 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:19.681590    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls podName:87973b43-d163-4c3d-975d-bf54b9ef4a55 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.181571951 +0000 UTC m=+36.596848111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls") pod "image-registry-5499cbfbf-zf8bj" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55") : secret "image-registry-tls" not found
Jul 13 19:22:19.681899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-tmp-dir\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.682009 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.681983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-certificates\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.682457 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.682432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-trusted-ca\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.685744 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.685723    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4ea392e-7d04-4c8a-bc09-ba81c768126b-cert\") pod \"ingress-canary-l5gft\" (UID: \"e4ea392e-7d04-4c8a-bc09-ba81c768126b\") " pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.685744 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.685731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-installation-pull-secrets\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.685880 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.685749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-image-registry-private-configuration\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.690239 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.690218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz7k4\" (UniqueName: \"kubernetes.io/projected/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-kube-api-access-tz7k4\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:19.691002 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.690982    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6c5s\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-kube-api-access-n6c5s\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.691098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.691079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-bound-sa-token\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:19.691157 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.691140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnggh\" (UniqueName: \"kubernetes.io/projected/e4ea392e-7d04-4c8a-bc09-ba81c768126b-kube-api-access-nnggh\") pod \"ingress-canary-l5gft\" (UID: \"e4ea392e-7d04-4c8a-bc09-ba81c768126b\") " pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:19.774707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:19.774681    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l5gft"
Jul 13 19:22:20.185144 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:20.185112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:20.185765 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:20.185187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:20.185765 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:20.185294    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:20.185765 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:20.185373    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls podName:ab74d8e7-399e-431b-8fbe-d57c1c8a9a59 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:21.185352603 +0000 UTC m=+37.600628779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls") pod "dns-default-llqzj" (UID: "ab74d8e7-399e-431b-8fbe-d57c1c8a9a59") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:20.185765 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:20.185301    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:20.185765 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:20.185401    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5499cbfbf-zf8bj: secret "image-registry-tls" not found
Jul 13 19:22:20.185765 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:20.185450    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls podName:87973b43-d163-4c3d-975d-bf54b9ef4a55 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:21.18543675 +0000 UTC m=+37.600712911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls") pod "image-registry-5499cbfbf-zf8bj" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55") : secret "image-registry-tls" not found
Jul 13 19:22:20.947146 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:20.947002    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l5gft"]
Jul 13 19:22:20.950766 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:20.950733    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ea392e_7d04_4c8a_bc09_ba81c768126b.slice/crio-1ba3eda717861e9edb483ce4eab605ab81d27d2ce7b516bdbbee43b723989fc4 WatchSource:0}: Error finding container 1ba3eda717861e9edb483ce4eab605ab81d27d2ce7b516bdbbee43b723989fc4: Status 404 returned error can't find the container with id 1ba3eda717861e9edb483ce4eab605ab81d27d2ce7b516bdbbee43b723989fc4
Jul 13 19:22:21.086080 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.086010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:21.086158 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.086010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:21.086206 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.086010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:21.090302 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.089440    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 19:22:21.090302 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.090033    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 19:22:21.090302 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.090058    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztp4q\""
Jul 13 19:22:21.090302 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.090279    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 19:22:21.090561 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.090424    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fp668\""
Jul 13 19:22:21.090614 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.090579    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 19:22:21.194123 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.194103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:21.194494 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.194152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:21.194494 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:21.194237    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:21.194494 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:21.194249    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:21.194494 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:21.194261    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5499cbfbf-zf8bj: secret "image-registry-tls" not found
Jul 13 19:22:21.194494 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:21.194285    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls podName:ab74d8e7-399e-431b-8fbe-d57c1c8a9a59 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:23.19427194 +0000 UTC m=+39.609548100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls") pod "dns-default-llqzj" (UID: "ab74d8e7-399e-431b-8fbe-d57c1c8a9a59") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:21.194494 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:21.194339    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls podName:87973b43-d163-4c3d-975d-bf54b9ef4a55 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:23.194324002 +0000 UTC m=+39.609600161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls") pod "image-registry-5499cbfbf-zf8bj" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55") : secret "image-registry-tls" not found
Jul 13 19:22:21.244299 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.244275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l5gft" event={"ID":"e4ea392e-7d04-4c8a-bc09-ba81c768126b","Type":"ContainerStarted","Data":"1ba3eda717861e9edb483ce4eab605ab81d27d2ce7b516bdbbee43b723989fc4"}
Jul 13 19:22:21.247197 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.247177    2562 generic.go:356] "Generic (PLEG): container finished" podID="530fff59-f369-4bf4-a159-837593b90da5" containerID="891c45521c010c35a90eca6b917026de7bf6386c20f8f0e869d20faab137db75" exitCode=0
Jul 13 19:22:21.247290 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:21.247216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerDied","Data":"891c45521c010c35a90eca6b917026de7bf6386c20f8f0e869d20faab137db75"}
Jul 13 19:22:22.253234 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.253195    2562 generic.go:356] "Generic (PLEG): container finished" podID="530fff59-f369-4bf4-a159-837593b90da5" containerID="276d29bf0504abed56fedbfbe9aa75348095a355c0ca4487773b73d188ff180e" exitCode=0
Jul 13 19:22:22.253646 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.253259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerDied","Data":"276d29bf0504abed56fedbfbe9aa75348095a355c0ca4487773b73d188ff180e"}
Jul 13 19:22:22.417270 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.417083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:22.593860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.593827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:22.749037 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.749003    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"]
Jul 13 19:22:22.779456 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.779417    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"]
Jul 13 19:22:22.779638 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.779581    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:22.782836 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.782449    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 13 19:22:22.782836 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.782507    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 13 19:22:22.782836 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.782746    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vklgp\""
Jul 13 19:22:22.783516 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.783495    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 13 19:22:22.783645 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.783582    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 13 19:22:22.793109 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.793085    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"]
Jul 13 19:22:22.793240 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.793221    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:22.795524 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.795506    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 13 19:22:22.807235 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.807214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"]
Jul 13 19:22:22.807336 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.807266    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"]
Jul 13 19:22:22.807336 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.807294    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"]
Jul 13 19:22:22.807475 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.807424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:22.810377 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.810339    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 13 19:22:22.810581 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.810569    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 13 19:22:22.811564 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.811524    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 13 19:22:22.811650 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.811619    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 13 19:22:22.907684 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/442f9a6e-34a8-4305-a16c-fb1c0af8d4ef-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b5f84b458-lmg7q\" (UID: \"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:22.907684 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ck9v\" (UniqueName: \"kubernetes.io/projected/0f9bc880-3f36-432a-acce-d4db187c1270-kube-api-access-8ck9v\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-ca\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wclt\" (UniqueName: \"kubernetes.io/projected/442f9a6e-34a8-4305-a16c-fb1c0af8d4ef-kube-api-access-5wclt\") pod \"managed-serviceaccount-addon-agent-b5f84b458-lmg7q\" (UID: \"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-hub\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f9bc880-3f36-432a-acce-d4db187c1270-tmp\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgbpn\" (UniqueName: \"kubernetes.io/projected/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-kube-api-access-vgbpn\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f9bc880-3f36-432a-acce-d4db187c1270-klusterlet-config\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:22.907886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:22.908166 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:22.908166 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:22.907913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.009195 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/442f9a6e-34a8-4305-a16c-fb1c0af8d4ef-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b5f84b458-lmg7q\" (UID: \"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:23.009357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ck9v\" (UniqueName: \"kubernetes.io/projected/0f9bc880-3f36-432a-acce-d4db187c1270-kube-api-access-8ck9v\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.009357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-ca\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.009357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009315    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wclt\" (UniqueName: \"kubernetes.io/projected/442f9a6e-34a8-4305-a16c-fb1c0af8d4ef-kube-api-access-5wclt\") pod \"managed-serviceaccount-addon-agent-b5f84b458-lmg7q\" (UID: \"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:23.009357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-hub\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.009357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f9bc880-3f36-432a-acce-d4db187c1270-tmp\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.009654 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgbpn\" (UniqueName: \"kubernetes.io/projected/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-kube-api-access-vgbpn\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.009654 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f9bc880-3f36-432a-acce-d4db187c1270-klusterlet-config\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.009856 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.009901 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.009930 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.010017 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.009838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f9bc880-3f36-432a-acce-d4db187c1270-tmp\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.010907 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.010700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.014849 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.014802    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.014849 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.014830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f9bc880-3f36-432a-acce-d4db187c1270-klusterlet-config\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.015018 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.014900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/442f9a6e-34a8-4305-a16c-fb1c0af8d4ef-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b5f84b458-lmg7q\" (UID: \"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:23.015410 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.015390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-hub\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.015410 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.015401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-ca\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.015567 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.015460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.019038 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.019019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgbpn\" (UniqueName: \"kubernetes.io/projected/76e9d114-e8f3-41e0-9cb7-2143c0eb80a6-kube-api-access-vgbpn\") pod \"cluster-proxy-proxy-agent-bf69656cd-9djg9\" (UID: \"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.019269 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.019248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wclt\" (UniqueName: \"kubernetes.io/projected/442f9a6e-34a8-4305-a16c-fb1c0af8d4ef-kube-api-access-5wclt\") pod \"managed-serviceaccount-addon-agent-b5f84b458-lmg7q\" (UID: \"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:23.019339 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.019306    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ck9v\" (UniqueName: \"kubernetes.io/projected/0f9bc880-3f36-432a-acce-d4db187c1270-kube-api-access-8ck9v\") pod \"klusterlet-addon-workmgr-646d97774b-nr7v7\" (UID: \"0f9bc880-3f36-432a-acce-d4db187c1270\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.103692 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.103656    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"
Jul 13 19:22:23.111151 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.111128    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:23.119042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.119016    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:22:23.212150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.212120    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:23.212268 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.212191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:23.212336 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:23.212281    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:23.212336 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:23.212319    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:23.212433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:23.212336    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5499cbfbf-zf8bj: secret "image-registry-tls" not found
Jul 13 19:22:23.212433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:23.212357    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls podName:ab74d8e7-399e-431b-8fbe-d57c1c8a9a59 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:27.212333874 +0000 UTC m=+43.627610051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls") pod "dns-default-llqzj" (UID: "ab74d8e7-399e-431b-8fbe-d57c1c8a9a59") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:23.212433 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:23.212398    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls podName:87973b43-d163-4c3d-975d-bf54b9ef4a55 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:27.212381685 +0000 UTC m=+43.627657868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls") pod "image-registry-5499cbfbf-zf8bj" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55") : secret "image-registry-tls" not found
Jul 13 19:22:23.258297 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.258271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" event={"ID":"530fff59-f369-4bf4-a159-837593b90da5","Type":"ContainerStarted","Data":"6df399644c2fbaf31276104e925de3f914d48902051d3034fc5035a4452264fd"}
Jul 13 19:22:23.284183 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.284138    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hcpgl" podStartSLOduration=8.614988674 podStartE2EDuration="39.284121313s" totalImagesPullingTime="30.669132639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:23.282436719 +0000 UTC m=+39.697712900" watchObservedRunningTime="2026-07-13 19:22:23.284121313 +0000 UTC m=+39.699397474"
Jul 13 19:22:23.405290 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.405261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:23.462793 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.462764    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q"]
Jul 13 19:22:23.468600 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.468526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:23.472054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.472036    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"]
Jul 13 19:22:23.477327 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.477305    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"]
Jul 13 19:22:23.525675 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:23.525645    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod442f9a6e_34a8_4305_a16c_fb1c0af8d4ef.slice/crio-6fab39bde33dc6294beca22dc836a3c2707c90c3a917171a75d358d4a6dafca7 WatchSource:0}: Error finding container 6fab39bde33dc6294beca22dc836a3c2707c90c3a917171a75d358d4a6dafca7: Status 404 returned error can't find the container with id 6fab39bde33dc6294beca22dc836a3c2707c90c3a917171a75d358d4a6dafca7
Jul 13 19:22:23.526675 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:23.526647    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76e9d114_e8f3_41e0_9cb7_2143c0eb80a6.slice/crio-a099915fbeb06d82d5841b39a9ee40f9c4f066e00a5f338e3d053e991b043dc1 WatchSource:0}: Error finding container a099915fbeb06d82d5841b39a9ee40f9c4f066e00a5f338e3d053e991b043dc1: Status 404 returned error can't find the container with id a099915fbeb06d82d5841b39a9ee40f9c4f066e00a5f338e3d053e991b043dc1
Jul 13 19:22:23.527382 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:23.527342    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f9bc880_3f36_432a_acce_d4db187c1270.slice/crio-3c04f1f68a5fecbba041cd3e540cbc614ce9e3e03db9fda1a3c702cf7d69cb35 WatchSource:0}: Error finding container 3c04f1f68a5fecbba041cd3e540cbc614ce9e3e03db9fda1a3c702cf7d69cb35: Status 404 returned error can't find the container with id 3c04f1f68a5fecbba041cd3e540cbc614ce9e3e03db9fda1a3c702cf7d69cb35
Jul 13 19:22:23.881247 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.881216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:23.981625 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:23.981599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:24.261954 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:24.261869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q" event={"ID":"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef","Type":"ContainerStarted","Data":"6fab39bde33dc6294beca22dc836a3c2707c90c3a917171a75d358d4a6dafca7"}
Jul 13 19:22:24.263511 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:24.263479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l5gft" event={"ID":"e4ea392e-7d04-4c8a-bc09-ba81c768126b","Type":"ContainerStarted","Data":"858fdc793ce85692a8d24f221a79483bff09f484fab79892168cbac7622c679d"}
Jul 13 19:22:24.264889 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:24.264860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" event={"ID":"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6","Type":"ContainerStarted","Data":"a099915fbeb06d82d5841b39a9ee40f9c4f066e00a5f338e3d053e991b043dc1"}
Jul 13 19:22:24.266099 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:24.266054    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7" event={"ID":"0f9bc880-3f36-432a-acce-d4db187c1270","Type":"ContainerStarted","Data":"3c04f1f68a5fecbba041cd3e540cbc614ce9e3e03db9fda1a3c702cf7d69cb35"}
Jul 13 19:22:24.280014 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:24.279970    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l5gft" podStartSLOduration=2.691513483 podStartE2EDuration="5.279957301s" totalImagesPullingTime="2.588443818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:24.278899598 +0000 UTC m=+40.694175780" watchObservedRunningTime="2026-07-13 19:22:24.279957301 +0000 UTC m=+40.695233483"
Jul 13 19:22:26.394114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:26.394079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:26.492763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:26.492690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:26.563762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:26.563728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:26.803373 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:26.803263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:26.904318 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:26.904289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:27.249385 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:27.249355    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:27.249569 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:27.249453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:27.249569 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:27.249523    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:27.249569 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:27.249570    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5499cbfbf-zf8bj: secret "image-registry-tls" not found
Jul 13 19:22:27.249726 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:27.249580    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:27.249726 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:27.249637    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls podName:87973b43-d163-4c3d-975d-bf54b9ef4a55 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:35.249618339 +0000 UTC m=+51.664894505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls") pod "image-registry-5499cbfbf-zf8bj" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55") : secret "image-registry-tls" not found
Jul 13 19:22:27.249726 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:27.249655    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls podName:ab74d8e7-399e-431b-8fbe-d57c1c8a9a59 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:35.249647859 +0000 UTC m=+51.664924023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls") pod "dns-default-llqzj" (UID: "ab74d8e7-399e-431b-8fbe-d57c1c8a9a59") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:28.724975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.724935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:28.788309 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.788285    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ckn56"]
Jul 13 19:22:28.803779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.803756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.803918 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.803883    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ckn56"]
Jul 13 19:22:28.814819 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.814787    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 19:22:28.815099 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.815083    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 19:22:28.815304 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.815281    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dkwp7\""
Jul 13 19:22:28.817974 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.816519    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 19:22:28.817974 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.816728    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 19:22:28.818925 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.818897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:28.860391 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.860364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prb55\" (UniqueName: \"kubernetes.io/projected/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-api-access-prb55\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.860508 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.860402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.860508 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.860428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-crio-socket\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.860619 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.860568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-data-volume\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.860619 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.860604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.877329 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.877295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:28.960969 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.960942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb55\" (UniqueName: \"kubernetes.io/projected/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-api-access-prb55\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.961115 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.960977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.961115 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.960999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-crio-socket\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.961115 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:28.961101    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:28.961282 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:28.961165    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:29.461145778 +0000 UTC m=+45.876421955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:28.961282 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.961170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-crio-socket\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.961282 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.961235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-data-volume\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.961282 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.961254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.961495 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:28.961403    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:28.961495 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:28.961455    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:29.461441263 +0000 UTC m=+45.876717439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:28.961633 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.961615    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-data-volume\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:28.969716 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:28.969697    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prb55\" (UniqueName: \"kubernetes.io/projected/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-api-access-prb55\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:29.465693 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:29.465657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:29.465977 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:29.465733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:29.465977 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:29.465792    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:29.465977 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:29.465875    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.465858875 +0000 UTC m=+46.881135036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:29.465977 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:29.465902    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:29.465977 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:29.465976    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.465962516 +0000 UTC m=+46.881238676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:29.647207 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:29.647166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:29.702975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:29.702939    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:29.781475 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:29.781406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:29.878779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:29.878747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:30.085504 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.085474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:30.216165 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.216135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:30.359500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.359440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:30.459071 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.459049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:30.474057 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.474036    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:30.474142 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.474117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:30.474190 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:30.474174    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:30.474190 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:30.474184    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:30.474254 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:30.474234    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:32.47421902 +0000 UTC m=+48.889495185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:30.474254 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:30.474247    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:32.474241605 +0000 UTC m=+48.889517769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:30.806517 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.806486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:30.875098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.875043    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6cg2s_94e22fbe-3c4c-4c65-946a-a6a2be93b2b7/dns-node-resolver/0.log"
Jul 13 19:22:30.904959 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:30.904931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:31.287842 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.287757    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q" event={"ID":"442f9a6e-34a8-4305-a16c-fb1c0af8d4ef","Type":"ContainerStarted","Data":"c96bf083499258247f54df4e2c9cf4634e1f8d0a865e35452ce6925de1eeb4bb"}
Jul 13 19:22:31.289863 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.289839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" event={"ID":"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6","Type":"ContainerStarted","Data":"0a6ef29a869f71c78978fb60cda6b2a8c856b2ecd18d7dbb3d17629fb87db651"}
Jul 13 19:22:31.289997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.289870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" event={"ID":"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6","Type":"ContainerStarted","Data":"7329208b206a491648f0980b48477076040d2f57a40aee6e808659f0f3027ec1"}
Jul 13 19:22:31.289997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.289883    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" event={"ID":"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6","Type":"ContainerStarted","Data":"541c447b6ba6fab9788e50e8214d1aaf01adc4a577f7bd7a1fd5a71bcbf91ecd"}
Jul 13 19:22:31.291324 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.291303    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7" event={"ID":"0f9bc880-3f36-432a-acce-d4db187c1270","Type":"ContainerStarted","Data":"9008c33a3612fe2ded91407ee5fc3bb5416babbd725691fc89bf26d2bb748665"}
Jul 13 19:22:31.291546 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.291513    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:31.293261 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.293243    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7"
Jul 13 19:22:31.302960 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.302920    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5f84b458-lmg7q" podStartSLOduration=2.451012327 podStartE2EDuration="9.302908263s" totalImagesPullingTime="6.851895936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:31.302761302 +0000 UTC m=+47.718037484" watchObservedRunningTime="2026-07-13 19:22:31.302908263 +0000 UTC m=+47.718184445"
Jul 13 19:22:31.323751 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.323716    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" podStartSLOduration=2.455969994 podStartE2EDuration="9.323704448s" totalImagesPullingTime="6.867734454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:31.323701376 +0000 UTC m=+47.738977558" watchObservedRunningTime="2026-07-13 19:22:31.323704448 +0000 UTC m=+47.738980630"
Jul 13 19:22:31.340554 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.340497    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-646d97774b-nr7v7" podStartSLOduration=2.488232463 podStartE2EDuration="9.34048976s" totalImagesPullingTime="6.852257297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:31.340223312 +0000 UTC m=+47.755499497" watchObservedRunningTime="2026-07-13 19:22:31.34048976 +0000 UTC m=+47.755765941"
Jul 13 19:22:31.873971 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.873944    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n5b2j_8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071/node-ca/0.log"
Jul 13 19:22:31.992813 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:31.992784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:32.488726 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:32.488692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:32.488919 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:32.488747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:32.488919 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:32.488817    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:32.488919 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:32.488887    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:32.488919 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:32.488911    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:36.488877681 +0000 UTC m=+52.904153855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:32.489131 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:32.488933    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:36.488920353 +0000 UTC m=+52.904196512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:32.543232 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:32.543203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:32.645975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:32.645940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:32.874605 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:32.874571    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l5gft_e4ea392e-7d04-4c8a-bc09-ba81c768126b/serve-healthcheck-canary/0.log"
Jul 13 19:22:32.882254 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:32.882230    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:33.872568 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:33.872519    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:35.309272 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:35.309238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:35.309659 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:35.309293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:35.309659 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:35.309387    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:35.309659 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:35.309401    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5499cbfbf-zf8bj: secret "image-registry-tls" not found
Jul 13 19:22:35.309659 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:35.309403    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:35.309659 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:35.309456    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls podName:87973b43-d163-4c3d-975d-bf54b9ef4a55 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:51.309441674 +0000 UTC m=+67.724717833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls") pod "image-registry-5499cbfbf-zf8bj" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55") : secret "image-registry-tls" not found
Jul 13 19:22:35.309659 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:35.309470    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls podName:ab74d8e7-399e-431b-8fbe-d57c1c8a9a59 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:51.309464623 +0000 UTC m=+67.724740783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls") pod "dns-default-llqzj" (UID: "ab74d8e7-399e-431b-8fbe-d57c1c8a9a59") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:35.712675 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:35.712600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:36.480690 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:36.480656    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:36.517818 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:36.517792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:36.517921 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:36.517858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:36.517959 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:36.517927    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:36.517991 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:36.517979    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls podName:7b1d8e47-12e4-4dad-b07d-6dadfb2ba146 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:44.517966179 +0000 UTC m=+60.933242339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ckn56" (UID: "7b1d8e47-12e4-4dad-b07d-6dadfb2ba146") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:36.518325 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:36.518309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:36.653273 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:36.653245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:36.751875 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:36.751801    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:37.565244 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:37.565203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:37.739232 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:37.739200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:38.138318 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:38.138289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:38.349787 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:38.349758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:38.534853 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:38.534787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:38.948687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:38.948657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:40.523317 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:40.523278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:40.579367 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:40.579340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:40.960393 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:40.960365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:41.027503 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:41.027479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:41.536744 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:41.536713    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.144844 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.144809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.245712 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.245684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.346279 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.346251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.409155 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.409092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.551900 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.551876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.742882 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.742801    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:42.909152 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:42.909125    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:43.762852 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:43.762813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:44.136963 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.136932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:44.236713 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.236684    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mw2d8"
Jul 13 19:22:44.334636 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.334597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:44.548302 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.548222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:44.580476 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.580446    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:44.582849 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.582831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b1d8e47-12e4-4dad-b07d-6dadfb2ba146-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ckn56\" (UID: \"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146\") " pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:44.728945 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.728918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dkwp7\""
Jul 13 19:22:44.737064 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.737046    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ckn56"
Jul 13 19:22:44.765001 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.764973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:44.893125 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:44.893097    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ckn56"]
Jul 13 19:22:44.896639 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:44.896615    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1d8e47_12e4_4dad_b07d_6dadfb2ba146.slice/crio-3ad0fe9cc0d7c7767a349f61fd405df3d45652fefa3b392f3f7299aa1ab75ab8 WatchSource:0}: Error finding container 3ad0fe9cc0d7c7767a349f61fd405df3d45652fefa3b392f3f7299aa1ab75ab8: Status 404 returned error can't find the container with id 3ad0fe9cc0d7c7767a349f61fd405df3d45652fefa3b392f3f7299aa1ab75ab8
Jul 13 19:22:45.232604 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:45.232504    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:45.333803 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:45.333768    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ckn56" event={"ID":"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146","Type":"ContainerStarted","Data":"3ad0fe9cc0d7c7767a349f61fd405df3d45652fefa3b392f3f7299aa1ab75ab8"}
Jul 13 19:22:45.756185 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:45.756157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:46.136912 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:46.136883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:46.331951 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:46.331921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:46.537663 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:46.537574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:46.742926 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:46.742888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:47.159671 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:47.159635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:47.260663 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:47.260622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:49.723120 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.723090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:49.723453 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.723159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:49.725783 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.725765    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 19:22:49.726713 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.726696    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 19:22:49.737496 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.736634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e22396dc-eaa1-48db-b917-bfea50c1c49c-original-pull-secret\") pod \"global-pull-secret-syncer-276kq\" (UID: \"e22396dc-eaa1-48db-b917-bfea50c1c49c\") " pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:49.737496 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.736690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec51aa55-8fdc-41a9-bc13-420515da094b-metrics-certs\") pod \"network-metrics-daemon-lnzkp\" (UID: \"ec51aa55-8fdc-41a9-bc13-420515da094b\") " pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:49.898451 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.898425    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-276kq"
Jul 13 19:22:49.911266 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.911240    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fp668\""
Jul 13 19:22:49.919282 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.919252    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnzkp"
Jul 13 19:22:49.924795 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.924769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:49.927644 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.927622    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 19:22:49.938320 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.938292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 19:22:49.950328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:49.950302    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c5jl\" (UniqueName: \"kubernetes.io/projected/417eafcd-0358-421d-8d0e-f6e5960c0352-kube-api-access-4c5jl\") pod \"network-check-target-twhgh\" (UID: \"417eafcd-0358-421d-8d0e-f6e5960c0352\") " pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:50.029412 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.029384    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-276kq"]
Jul 13 19:22:50.033352 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:50.033323    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22396dc_eaa1_48db_b917_bfea50c1c49c.slice/crio-690786b0c59160136f7376ccaf352f283de0289eba2c3fa9754644a3acb55b1c WatchSource:0}: Error finding container 690786b0c59160136f7376ccaf352f283de0289eba2c3fa9754644a3acb55b1c: Status 404 returned error can't find the container with id 690786b0c59160136f7376ccaf352f283de0289eba2c3fa9754644a3acb55b1c
Jul 13 19:22:50.050755 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.050731    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lnzkp"]
Jul 13 19:22:50.055469 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:50.055437    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec51aa55_8fdc_41a9_bc13_420515da094b.slice/crio-ea48392f2dc35c03d0b791c3cf2dffb4a92e22162edd5c589d848e1832954a30 WatchSource:0}: Error finding container ea48392f2dc35c03d0b791c3cf2dffb4a92e22162edd5c589d848e1832954a30: Status 404 returned error can't find the container with id ea48392f2dc35c03d0b791c3cf2dffb4a92e22162edd5c589d848e1832954a30
Jul 13 19:22:50.206375 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.206356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztp4q\""
Jul 13 19:22:50.213844 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.213815    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:50.334329 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.334302    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-twhgh"]
Jul 13 19:22:50.336906 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:50.336878    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417eafcd_0358_421d_8d0e_f6e5960c0352.slice/crio-3d03ddc19f384e78b1d01fece567203bc7608726c132ce81bc5388424ba6807a WatchSource:0}: Error finding container 3d03ddc19f384e78b1d01fece567203bc7608726c132ce81bc5388424ba6807a: Status 404 returned error can't find the container with id 3d03ddc19f384e78b1d01fece567203bc7608726c132ce81bc5388424ba6807a
Jul 13 19:22:50.349642 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.349614    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ckn56" event={"ID":"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146","Type":"ContainerStarted","Data":"7849f3ca1236ecd4483f470f9191c255529027155abe25a6034a1d121ba4dd5a"}
Jul 13 19:22:50.349745 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.349670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ckn56" event={"ID":"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146","Type":"ContainerStarted","Data":"832f846e05daec8907da92b573f5cfadf4c53c8da4ce00342222134cf9062571"}
Jul 13 19:22:50.350769 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.350748    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-twhgh" event={"ID":"417eafcd-0358-421d-8d0e-f6e5960c0352","Type":"ContainerStarted","Data":"3d03ddc19f384e78b1d01fece567203bc7608726c132ce81bc5388424ba6807a"}
Jul 13 19:22:50.351865 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.351844    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnzkp" event={"ID":"ec51aa55-8fdc-41a9-bc13-420515da094b","Type":"ContainerStarted","Data":"ea48392f2dc35c03d0b791c3cf2dffb4a92e22162edd5c589d848e1832954a30"}
Jul 13 19:22:50.352821 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.352804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-276kq" event={"ID":"e22396dc-eaa1-48db-b917-bfea50c1c49c","Type":"ContainerStarted","Data":"690786b0c59160136f7376ccaf352f283de0289eba2c3fa9754644a3acb55b1c"}
Jul 13 19:22:50.812709 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.812672    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5499cbfbf-zf8bj"]
Jul 13 19:22:50.813109 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:22:50.812914    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj" podUID="87973b43-d163-4c3d-975d-bf54b9ef4a55"
Jul 13 19:22:50.891030 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.890998    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:50.982480 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:50.982415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:51.337462 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.337387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:51.337462 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.337449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:51.341810 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.341783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab74d8e7-399e-431b-8fbe-d57c1c8a9a59-metrics-tls\") pod \"dns-default-llqzj\" (UID: \"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59\") " pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:51.348508 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.348461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"image-registry-5499cbfbf-zf8bj\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") " pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:51.356219 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.356187    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:51.362963 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.361995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:51.362963 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.362683    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnzkp" event={"ID":"ec51aa55-8fdc-41a9-bc13-420515da094b","Type":"ContainerStarted","Data":"257b83a15e77eb9cfd5399cf3712412bb970a5b4738feb5febad7458ee01662f"}
Jul 13 19:22:51.362963 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.362732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnzkp" event={"ID":"ec51aa55-8fdc-41a9-bc13-420515da094b","Type":"ContainerStarted","Data":"007645b0f96598cea91ced58e98eb97375d00baa6d99b055b984c12f5a352984"}
Jul 13 19:22:51.369113 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.369006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:51.384805 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.384759    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lnzkp" podStartSLOduration=66.481653958 podStartE2EDuration="1m7.384741485s" totalImagesPullingTime="903.087527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:51.3833243 +0000 UTC m=+67.798600570" watchObservedRunningTime="2026-07-13 19:22:51.384741485 +0000 UTC m=+67.800017670"
Jul 13 19:22:51.438391 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438364    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6c5s\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-kube-api-access-n6c5s\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438528 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438406    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-bound-sa-token\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438528 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-installation-pull-secrets\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438528 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438464    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-certificates\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438528 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438508    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-trusted-ca\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438585    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87973b43-d163-4c3d-975d-bf54b9ef4a55-ca-trust-extracted\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.438762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.438643    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-image-registry-private-configuration\") pod \"87973b43-d163-4c3d-975d-bf54b9ef4a55\" (UID: \"87973b43-d163-4c3d-975d-bf54b9ef4a55\") "
Jul 13 19:22:51.440495 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.440448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:22:51.440666 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.440481    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:22:51.440666 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.440587    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87973b43-d163-4c3d-975d-bf54b9ef4a55-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:22:51.443606 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.443575    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:22:51.443969 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.443927    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:22:51.444298 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.444253    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:22:51.446191 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.446162    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-kube-api-access-n6c5s" (OuterVolumeSpecName: "kube-api-access-n6c5s") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "kube-api-access-n6c5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:22:51.446423 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.446400    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "87973b43-d163-4c3d-975d-bf54b9ef4a55" (UID: "87973b43-d163-4c3d-975d-bf54b9ef4a55"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:22:51.458621 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.458598    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539790    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6c5s\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-kube-api-access-n6c5s\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539819    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-bound-sa-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539834    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-installation-pull-secrets\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539849    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-certificates\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539862    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87973b43-d163-4c3d-975d-bf54b9ef4a55-registry-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539875    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87973b43-d163-4c3d-975d-bf54b9ef4a55-trusted-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539889    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87973b43-d163-4c3d-975d-bf54b9ef4a55-ca-trust-extracted\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.539934 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.539901    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87973b43-d163-4c3d-975d-bf54b9ef4a55-image-registry-private-configuration\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:22:51.564889 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.564860    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mf9hk\""
Jul 13 19:22:51.572145 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.572128    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:51.639067 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.638926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:51.710005 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.709968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:51.747946 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:51.747917    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-llqzj"]
Jul 13 19:22:51.753516 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:22:51.753488    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab74d8e7_399e_431b_8fbe_d57c1c8a9a59.slice/crio-83cc1331891b5efdcfd5ba63c5f1b5fee9ba2ef46ad2ca6368aa512798c7d7fb WatchSource:0}: Error finding container 83cc1331891b5efdcfd5ba63c5f1b5fee9ba2ef46ad2ca6368aa512798c7d7fb: Status 404 returned error can't find the container with id 83cc1331891b5efdcfd5ba63c5f1b5fee9ba2ef46ad2ca6368aa512798c7d7fb
Jul 13 19:22:52.222515 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.222453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:52.368867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.368342    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5499cbfbf-zf8bj"
Jul 13 19:22:52.368867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.368348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llqzj" event={"ID":"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59","Type":"ContainerStarted","Data":"83cc1331891b5efdcfd5ba63c5f1b5fee9ba2ef46ad2ca6368aa512798c7d7fb"}
Jul 13 19:22:52.410288 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.409624    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5499cbfbf-zf8bj"]
Jul 13 19:22:52.412248 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.412219    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5499cbfbf-zf8bj"]
Jul 13 19:22:52.655304 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.655171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:52.844077 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:52.844043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:53.614491 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:53.614452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:54.012333 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:54.012246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:54.091168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:54.091137    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87973b43-d163-4c3d-975d-bf54b9ef4a55" path="/var/lib/kubelet/pods/87973b43-d163-4c3d-975d-bf54b9ef4a55/volumes"
Jul 13 19:22:54.111637 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:54.111603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:57.387876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.387832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-twhgh" event={"ID":"417eafcd-0358-421d-8d0e-f6e5960c0352","Type":"ContainerStarted","Data":"2fd4bdeb6ee7c680e24262cb30928c3b310d5f12114be8797367d3d83b0ed988"}
Jul 13 19:22:57.388344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.387991    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:22:57.389352 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.389328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-276kq" event={"ID":"e22396dc-eaa1-48db-b917-bfea50c1c49c","Type":"ContainerStarted","Data":"dde41ab9f20860af4ce4b89a248c464de627e0c5feff034f69a9fa66a3bb537f"}
Jul 13 19:22:57.391106 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.391084    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llqzj" event={"ID":"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59","Type":"ContainerStarted","Data":"59239132827c3f95186941a340d720fbe8258d00f6a8cf3c9f81a0ad55a49203"}
Jul 13 19:22:57.391199 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.391128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llqzj" event={"ID":"ab74d8e7-399e-431b-8fbe-d57c1c8a9a59","Type":"ContainerStarted","Data":"0ba155b1d74aee99c343129d870759e34210e77dab2b6f0c54ddf7b799f93182"}
Jul 13 19:22:57.391301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.391282    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-llqzj"
Jul 13 19:22:57.393073 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.393052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ckn56" event={"ID":"7b1d8e47-12e4-4dad-b07d-6dadfb2ba146","Type":"ContainerStarted","Data":"666e57f6e60c7334786f1e12a702fa97f7b6a76981105449644558b917dd83ff"}
Jul 13 19:22:57.403763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.403719    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-twhgh" podStartSLOduration=67.006435717 podStartE2EDuration="1m13.403707751s" totalImagesPullingTime="6.397272034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:57.40308726 +0000 UTC m=+73.818363443" watchObservedRunningTime="2026-07-13 19:22:57.403707751 +0000 UTC m=+73.818983933"
Jul 13 19:22:57.422959 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.422912    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ckn56" podStartSLOduration=17.68854713 podStartE2EDuration="29.4228956s" totalImagesPullingTime="11.73434847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:22:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:57.421793133 +0000 UTC m=+73.837069315" watchObservedRunningTime="2026-07-13 19:22:57.4228956 +0000 UTC m=+73.838171783"
Jul 13 19:22:57.448271 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.448210    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-llqzj" podStartSLOduration=33.471195745 podStartE2EDuration="38.448195572s" totalImagesPullingTime="4.976999827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:57.448123471 +0000 UTC m=+73.863399667" watchObservedRunningTime="2026-07-13 19:22:57.448195572 +0000 UTC m=+73.863471755"
Jul 13 19:22:57.482879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:57.482837    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-276kq" podStartSLOduration=66.780670099 podStartE2EDuration="1m13.482823294s" totalImagesPullingTime="6.702153195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:57.482090968 +0000 UTC m=+73.897367150" watchObservedRunningTime="2026-07-13 19:22:57.482823294 +0000 UTC m=+73.898099476"
Jul 13 19:22:58.398740 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:58.398707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:58.485953 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:58.485927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:58.911193 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:58.911158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:58.967472 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:58.967445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:22:59.321256 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:22:59.321218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:01.609730 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:01.609689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:01.719479 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:01.719444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:01.820066 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:01.820033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:02.047858 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:02.047758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:02.110085 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:02.110049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:03.599464 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:03.599432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:03.691182 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:03.691152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:03.792640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:03.792606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:03.893362 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:03.893289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:04.084589 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:04.084527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:04.278047 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:04.277969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:04.355891 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:04.355859    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:04.655670 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:04.655562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:05.279228 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:05.279195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:05.379640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:05.379608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:05.480524 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:05.480491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:05.809121 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:05.809090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.001470 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.001435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.069268 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.069190    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g2rmr"]
Jul 13 19:23:06.071623 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.071608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.074228 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.074206    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 19:23:06.074364 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.074229    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 19:23:06.075249 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.075228    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 19:23:06.075447 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.075267    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 19:23:06.075447 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.075323    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 19:23:06.075447 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.075343    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gdkc8\""
Jul 13 19:23:06.075644 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.075460    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 19:23:06.102265 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.102243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.143950 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.143923    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-wtmp\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.143952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8efa588-1365-42d2-b35c-bb23fab7aa0b-metrics-client-ca\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.143981    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-sys\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.143996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-root\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.144035    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.144051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-tls\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.144067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144259 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.144083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ld25\" (UniqueName: \"kubernetes.io/projected/c8efa588-1365-42d2-b35c-bb23fab7aa0b-kube-api-access-4ld25\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.144259 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.144143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-textfile\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.201270 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.201236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.245468 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-sys\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245468 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-root\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-tls\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-sys\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-root\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ld25\" (UniqueName: \"kubernetes.io/projected/c8efa588-1365-42d2-b35c-bb23fab7aa0b-kube-api-access-4ld25\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-textfile\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-wtmp\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.245707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8efa588-1365-42d2-b35c-bb23fab7aa0b-metrics-client-ca\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.246133 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.245865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-wtmp\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.246133 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.246023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-textfile\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.246133 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.246120    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.246238 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.246192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8efa588-1365-42d2-b35c-bb23fab7aa0b-metrics-client-ca\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.248051 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.248027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.248154 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.248140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8efa588-1365-42d2-b35c-bb23fab7aa0b-node-exporter-tls\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.253785 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.253763    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ld25\" (UniqueName: \"kubernetes.io/projected/c8efa588-1365-42d2-b35c-bb23fab7aa0b-kube-api-access-4ld25\") pod \"node-exporter-g2rmr\" (UID: \"c8efa588-1365-42d2-b35c-bb23fab7aa0b\") " pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.305076 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.305049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.380882 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.380831    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g2rmr"
Jul 13 19:23:06.390003 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:23:06.389976    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8efa588_1365_42d2_b35c_bb23fab7aa0b.slice/crio-a9bc441ae37a68226113a9187b53512deee66ff230575656c7653b5016e123ff WatchSource:0}: Error finding container a9bc441ae37a68226113a9187b53512deee66ff230575656c7653b5016e123ff: Status 404 returned error can't find the container with id a9bc441ae37a68226113a9187b53512deee66ff230575656c7653b5016e123ff
Jul 13 19:23:06.406650 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.406628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.424884 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.424860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g2rmr" event={"ID":"c8efa588-1365-42d2-b35c-bb23fab7aa0b","Type":"ContainerStarted","Data":"a9bc441ae37a68226113a9187b53512deee66ff230575656c7653b5016e123ff"}
Jul 13 19:23:06.502397 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.502368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:06.864424 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:06.864395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:07.037869 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:07.037831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:07.140697 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:07.140604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:07.254061 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:07.254027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:07.383472 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:07.383450    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:07.398242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:07.398224    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-llqzj"
Jul 13 19:23:07.471034 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:07.471009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:08.432593 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:08.432558    2562 generic.go:356] "Generic (PLEG): container finished" podID="c8efa588-1365-42d2-b35c-bb23fab7aa0b" containerID="d0ca0031ec0e81474c2db8947164cde22a5b8ce118e715d5e391deec3c6fb3ac" exitCode=0
Jul 13 19:23:08.432954 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:08.432599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g2rmr" event={"ID":"c8efa588-1365-42d2-b35c-bb23fab7aa0b","Type":"ContainerDied","Data":"d0ca0031ec0e81474c2db8947164cde22a5b8ce118e715d5e391deec3c6fb3ac"}
Jul 13 19:23:08.986089 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:08.986056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:09.047001 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:09.046975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:09.360488 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:09.360460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:09.438869 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:09.438841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g2rmr" event={"ID":"c8efa588-1365-42d2-b35c-bb23fab7aa0b","Type":"ContainerStarted","Data":"05ae61cb7ce66ca4c94dca8bbc3df4d0b9f6cda6e5b96e166785995268e6be91"}
Jul 13 19:23:09.439215 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:09.438885    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g2rmr" event={"ID":"c8efa588-1365-42d2-b35c-bb23fab7aa0b","Type":"ContainerStarted","Data":"8016f0fd045daea257c2d6abc0bbf2b88b252f5210e1bc6154f47f6e8aebbd37"}
Jul 13 19:23:09.856641 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:09.856612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:10.828386 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:10.828355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:11.008078 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:11.008047    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:11.088911 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:11.088821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:12.319466 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:12.319430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:12.671867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:12.671785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:12.857470 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:12.857429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:14.477971 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:14.477938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:14.653213 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:14.653179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:14.770418 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:14.770338    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:15.085058 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:15.085027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:15.141008 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:15.140970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:16.654815 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:16.654784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:16.724923 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:16.724893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:17.082875 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:17.082846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:17.278652 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:17.278625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:17.495401 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:17.495321    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:17.595255 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:17.595224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:20.309013 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:20.308978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:20.418527 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:20.418494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:21.601627 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:21.601595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:21.703059 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:21.703028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:21.894394 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:21.894326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:21.950419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:21.950392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:22.119877 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:22.119846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:22.176001 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:22.175922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:24.089459 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:24.089425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:25.045040 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:25.045010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:25.142311 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:25.142275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:25.569477 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:25.569434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:25.668815 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:25.668770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:26.707493 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:26.707460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:26.808656 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:26.808623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:27.692618 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:27.692580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:27.792153 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:27.792124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:28.398450 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:28.398422    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-twhgh"
Jul 13 19:23:28.415594 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:28.415527    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g2rmr" podStartSLOduration=21.35699595 podStartE2EDuration="22.415514501s" totalImagesPullingTime="985.447444ms" totalInitContainerRuntime="73.071107ms" isStatefulPod=true podCreationTimestamp="2026-07-13 19:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:09.463515665 +0000 UTC m=+85.878791846" watchObservedRunningTime="2026-07-13 19:23:28.415514501 +0000 UTC m=+104.830790683"
Jul 13 19:23:30.755093 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:30.755059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:30.825516 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:30.825487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:31.734073 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:31.734044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:31.817003 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:31.816976    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:32.055926 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:32.055856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:32.150232 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:32.150206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:32.740695 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:32.740663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:33.041647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:33.041561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:47.061720 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:47.061685    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:47.135821 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:47.135796    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:23:53.979176 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:23:53.979129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:03.120523 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:03.120474    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" podUID="76e9d114-e8f3-41e0-9cb7-2143c0eb80a6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 13 19:24:03.405248 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:03.405161    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:13.120136 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:13.120096    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" podUID="76e9d114-e8f3-41e0-9cb7-2143c0eb80a6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 13 19:24:23.120604 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.120565    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" podUID="76e9d114-e8f3-41e0-9cb7-2143c0eb80a6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 13 19:24:23.121150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.120631    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9"
Jul 13 19:24:23.121150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.121055    2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"0a6ef29a869f71c78978fb60cda6b2a8c856b2ecd18d7dbb3d17629fb87db651"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 13 19:24:23.121150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.121101    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" podUID="76e9d114-e8f3-41e0-9cb7-2143c0eb80a6" containerName="service-proxy" containerID="cri-o://0a6ef29a869f71c78978fb60cda6b2a8c856b2ecd18d7dbb3d17629fb87db651" gracePeriod=30
Jul 13 19:24:23.677395 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.677363    2562 generic.go:356] "Generic (PLEG): container finished" podID="76e9d114-e8f3-41e0-9cb7-2143c0eb80a6" containerID="0a6ef29a869f71c78978fb60cda6b2a8c856b2ecd18d7dbb3d17629fb87db651" exitCode=2
Jul 13 19:24:23.677577 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.677437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" event={"ID":"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6","Type":"ContainerDied","Data":"0a6ef29a869f71c78978fb60cda6b2a8c856b2ecd18d7dbb3d17629fb87db651"}
Jul 13 19:24:23.677577 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:23.677479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf69656cd-9djg9" event={"ID":"76e9d114-e8f3-41e0-9cb7-2143c0eb80a6","Type":"ContainerStarted","Data":"02453e8aefaaafb3a5eae785d20d06ced2351b0691c8a4ec63f0d52e9c70365e"}
Jul 13 19:24:26.107659 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:26.107622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:26.162289 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:26.162262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:33.777557 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:33.777494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:33.835719 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:33.835681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:33.894593 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:33.894559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:34.489820 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:34.489782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:45.136337 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:45.136300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:45.228051 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:45.228012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:24:45.448971 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:24:45.448897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:25:04.634614 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:25:04.634577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:25:40.548928 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:25:40.548894    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:25:40.605914 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:25:40.605892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:25:52.430507 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:25:52.430469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:25:52.531236 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:25:52.531207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:02.050840 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:02.050798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:02.151339 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:02.151307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:03.863010 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:03.862976    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:04.332194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:04.332157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:04.433547 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:04.433514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:04.638788 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:04.638682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:04.695568 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:04.695531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:07.087203 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:07.087171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:14.469426 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:14.469386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:14.530964 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:14.530934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:43.915455 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:43.915428    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 19:26:53.701492 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:53.701451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:26:55.825903 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:26:55.825864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:25.959582 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:25.959529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:27.332386 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:27.332346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:29.777524 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:29.777492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:30.233419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:30.233385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:31.182714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:31.182681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:31.291293 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:31.291259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:32.086850 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:32.086812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:32.333191 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:32.333160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:34.464351 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:34.464314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:34.754810 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:34.754701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:36.567134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:36.567099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:36.646900 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:36.646873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:42.055285 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:42.055250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:43.861486 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.861452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:43.898978 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.898947    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4jhzv"]
Jul 13 19:27:43.901779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.901761    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:43.904027 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.904005    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wv2cx\""
Jul 13 19:27:43.904108 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.904043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 19:27:43.904941 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.904926    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 19:27:43.914978 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.914957    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4jhzv"]
Jul 13 19:27:43.961287 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:43.961250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:44.066611 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.066582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdm2c\" (UniqueName: \"kubernetes.io/projected/3571affa-09e2-48d5-b9b8-d5d08ec910df-kube-api-access-vdm2c\") pod \"cert-manager-55b4d954d5-4jhzv\" (UID: \"3571affa-09e2-48d5-b9b8-d5d08ec910df\") " pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.066768 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.066626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3571affa-09e2-48d5-b9b8-d5d08ec910df-bound-sa-token\") pod \"cert-manager-55b4d954d5-4jhzv\" (UID: \"3571affa-09e2-48d5-b9b8-d5d08ec910df\") " pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.167420 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.167344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3571affa-09e2-48d5-b9b8-d5d08ec910df-bound-sa-token\") pod \"cert-manager-55b4d954d5-4jhzv\" (UID: \"3571affa-09e2-48d5-b9b8-d5d08ec910df\") " pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.167577 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.167435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdm2c\" (UniqueName: \"kubernetes.io/projected/3571affa-09e2-48d5-b9b8-d5d08ec910df-kube-api-access-vdm2c\") pod \"cert-manager-55b4d954d5-4jhzv\" (UID: \"3571affa-09e2-48d5-b9b8-d5d08ec910df\") " pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.176094 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.176070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3571affa-09e2-48d5-b9b8-d5d08ec910df-bound-sa-token\") pod \"cert-manager-55b4d954d5-4jhzv\" (UID: \"3571affa-09e2-48d5-b9b8-d5d08ec910df\") " pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.176442 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.176421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdm2c\" (UniqueName: \"kubernetes.io/projected/3571affa-09e2-48d5-b9b8-d5d08ec910df-kube-api-access-vdm2c\") pod \"cert-manager-55b4d954d5-4jhzv\" (UID: \"3571affa-09e2-48d5-b9b8-d5d08ec910df\") " pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.210389 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.210367    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4jhzv"
Jul 13 19:27:44.335975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.335896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4jhzv"]
Jul 13 19:27:44.338392 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:27:44.338361    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3571affa_09e2_48d5_b9b8_d5d08ec910df.slice/crio-c6a254de9d54e809ac8fc36b5a50b1a8b0fab80637562d0f27ce65ad77aa0789 WatchSource:0}: Error finding container c6a254de9d54e809ac8fc36b5a50b1a8b0fab80637562d0f27ce65ad77aa0789: Status 404 returned error can't find the container with id c6a254de9d54e809ac8fc36b5a50b1a8b0fab80637562d0f27ce65ad77aa0789
Jul 13 19:27:44.340114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:44.340096    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:27:45.304378 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:45.304335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4jhzv" event={"ID":"3571affa-09e2-48d5-b9b8-d5d08ec910df","Type":"ContainerStarted","Data":"c6a254de9d54e809ac8fc36b5a50b1a8b0fab80637562d0f27ce65ad77aa0789"}
Jul 13 19:27:48.316522 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:48.316486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4jhzv" event={"ID":"3571affa-09e2-48d5-b9b8-d5d08ec910df","Type":"ContainerStarted","Data":"291e95058ffef94e7ab2b91fdb07104c1634d7bd909ce4bf5b0293f52498fbb4"}
Jul 13 19:27:48.333050 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:48.333012    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4jhzv" podStartSLOduration=2.153365452 podStartE2EDuration="5.332993852s" totalImagesPullingTime="3.1796284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:27:48.332788696 +0000 UTC m=+364.748064879" watchObservedRunningTime="2026-07-13 19:27:48.332993852 +0000 UTC m=+364.748270038"
Jul 13 19:27:49.547197 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:49.547164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:50.081819 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:50.081789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:27:51.379234 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:27:51.379201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:02.434278 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:02.434242    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:02.537210 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:02.537171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:07.613800 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:07.613769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:07.713406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:07.713378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:08.255940 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.255907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:08.597590 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.597555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:08.705754 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.705734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:08.737304 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.737279    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"]
Jul 13 19:28:08.740240 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.740219    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.744914 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.744882    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 19:28:08.745015 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.744910    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 19:28:08.745088 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.745019    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7pkj5\""
Jul 13 19:28:08.745348 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.745327    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 19:28:08.745424 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.745335    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 19:28:08.769442 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.769418    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"]
Jul 13 19:28:08.838398 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.838374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/067fd0f8-6a43-420c-a7db-09b881518dfa-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.838489 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.838404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/067fd0f8-6a43-420c-a7db-09b881518dfa-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.838489 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.838421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/067fd0f8-6a43-420c-a7db-09b881518dfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.838604 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.838511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/067fd0f8-6a43-420c-a7db-09b881518dfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.838604 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.838578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rbg8\" (UniqueName: \"kubernetes.io/projected/067fd0f8-6a43-420c-a7db-09b881518dfa-kube-api-access-4rbg8\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.902680 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.902615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:08.939290 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.939269    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/067fd0f8-6a43-420c-a7db-09b881518dfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.939371 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.939296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rbg8\" (UniqueName: \"kubernetes.io/projected/067fd0f8-6a43-420c-a7db-09b881518dfa-kube-api-access-4rbg8\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.939371 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.939327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/067fd0f8-6a43-420c-a7db-09b881518dfa-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.939478 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.939448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/067fd0f8-6a43-420c-a7db-09b881518dfa-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.939521 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.939492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/067fd0f8-6a43-420c-a7db-09b881518dfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.939763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.939746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/067fd0f8-6a43-420c-a7db-09b881518dfa-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.941739 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.941719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/067fd0f8-6a43-420c-a7db-09b881518dfa-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.941878 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.941861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/067fd0f8-6a43-420c-a7db-09b881518dfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.942079 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.942057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/067fd0f8-6a43-420c-a7db-09b881518dfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.949959 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.949941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rbg8\" (UniqueName: \"kubernetes.io/projected/067fd0f8-6a43-420c-a7db-09b881518dfa-kube-api-access-4rbg8\") pod \"opendatahub-operator-controller-manager-6996d85cfb-fgl2c\" (UID: \"067fd0f8-6a43-420c-a7db-09b881518dfa\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:08.999014 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:08.998997    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:09.050047 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.050027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:09.100385 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.100350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:09.183114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.183049    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"]
Jul 13 19:28:09.186207 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:28:09.186182    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067fd0f8_6a43_420c_a7db_09b881518dfa.slice/crio-cd6395721df09420b098526eb8ef182c71eccfdb0c8faba1a6714de8ef3344c5 WatchSource:0}: Error finding container cd6395721df09420b098526eb8ef182c71eccfdb0c8faba1a6714de8ef3344c5: Status 404 returned error can't find the container with id cd6395721df09420b098526eb8ef182c71eccfdb0c8faba1a6714de8ef3344c5
Jul 13 19:28:09.188932 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.188913    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:09.254051 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.254029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:09.365922 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.365900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:09.379301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:09.379279    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c" event={"ID":"067fd0f8-6a43-420c-a7db-09b881518dfa","Type":"ContainerStarted","Data":"cd6395721df09420b098526eb8ef182c71eccfdb0c8faba1a6714de8ef3344c5"}
Jul 13 19:28:13.393855 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:13.393819    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c" event={"ID":"067fd0f8-6a43-420c-a7db-09b881518dfa","Type":"ContainerStarted","Data":"64cc04b6aabdfba09989ff03f9d779af494fafcd0506463e7daa5dc0e263ad85"}
Jul 13 19:28:14.398027 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:14.397991    2562 generic.go:356] "Generic (PLEG): container finished" podID="067fd0f8-6a43-420c-a7db-09b881518dfa" containerID="64cc04b6aabdfba09989ff03f9d779af494fafcd0506463e7daa5dc0e263ad85" exitCode=0
Jul 13 19:28:14.398480 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:14.398063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c" event={"ID":"067fd0f8-6a43-420c-a7db-09b881518dfa","Type":"ContainerDied","Data":"64cc04b6aabdfba09989ff03f9d779af494fafcd0506463e7daa5dc0e263ad85"}
Jul 13 19:28:16.406809 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:16.406774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c" event={"ID":"067fd0f8-6a43-420c-a7db-09b881518dfa","Type":"ContainerStarted","Data":"4d08d05c8f291d0df69c86cd61d1f8acaf73b4f508333c9087b0710b96042292"}
Jul 13 19:28:16.407260 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:16.406911    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:17.615882 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:17.615849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:17.717136 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:17.717106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:20.666270 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:20.666238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:20.775149 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:20.775114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:26.036680 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:26.036643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:26.099922 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:26.099891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:27.412968 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:27.412941    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c"
Jul 13 19:28:27.436343 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:27.436254    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-fgl2c" podStartSLOduration=14.192974786 podStartE2EDuration="19.436237929s" totalImagesPullingTime="4.172263055s" totalInitContainerRuntime="1.071000088s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:16.432435276 +0000 UTC m=+392.847711457" watchObservedRunningTime="2026-07-13 19:28:27.436237929 +0000 UTC m=+403.851514112"
Jul 13 19:28:29.078295 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.078257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:29.270684 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.270655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:29.688640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.688614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:29.806331 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.806307    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"]
Jul 13 19:28:29.814219 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.814200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.817107 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.817089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 19:28:29.817215 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.817163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 19:28:29.818074 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.818060    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 19:28:29.820876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.820852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 19:28:29.826340 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.826319    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"]
Jul 13 19:28:29.868097 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.868071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcg96\" (UniqueName: \"kubernetes.io/projected/c3dea633-36df-46bf-a96c-55e0a58f7567-kube-api-access-tcg96\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.868203 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.868112    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3dea633-36df-46bf-a96c-55e0a58f7567-tls-certs\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.868203 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.868143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3dea633-36df-46bf-a96c-55e0a58f7567-tmp\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.868766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.868724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:29.968572 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.968467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcg96\" (UniqueName: \"kubernetes.io/projected/c3dea633-36df-46bf-a96c-55e0a58f7567-kube-api-access-tcg96\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.968572 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.968512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3dea633-36df-46bf-a96c-55e0a58f7567-tls-certs\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.968572 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.968565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3dea633-36df-46bf-a96c-55e0a58f7567-tmp\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.970910 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.970888    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3dea633-36df-46bf-a96c-55e0a58f7567-tmp\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.971215 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.971199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3dea633-36df-46bf-a96c-55e0a58f7567-tls-certs\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:29.981578 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:29.981530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcg96\" (UniqueName: \"kubernetes.io/projected/c3dea633-36df-46bf-a96c-55e0a58f7567-kube-api-access-tcg96\") pod \"kube-auth-proxy-55f6586c87-mxm6s\" (UID: \"c3dea633-36df-46bf-a96c-55e0a58f7567\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:30.123261 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.123232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"
Jul 13 19:28:30.124237 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.124211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:30.248385 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.248317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:30.266338 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.266314    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s"]
Jul 13 19:28:30.268548 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:28:30.268507    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dea633_36df_46bf_a96c_55e0a58f7567.slice/crio-c8133921b02064842fd7560eb7b0df972daa01d9dd385865427e7babbcd8366d WatchSource:0}: Error finding container c8133921b02064842fd7560eb7b0df972daa01d9dd385865427e7babbcd8366d: Status 404 returned error can't find the container with id c8133921b02064842fd7560eb7b0df972daa01d9dd385865427e7babbcd8366d
Jul 13 19:28:30.452698 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.452666    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s" event={"ID":"c3dea633-36df-46bf-a96c-55e0a58f7567","Type":"ContainerStarted","Data":"c8133921b02064842fd7560eb7b0df972daa01d9dd385865427e7babbcd8366d"}
Jul 13 19:28:30.852387 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.852350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:30.926649 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.926616    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6ljsh"]
Jul 13 19:28:30.929617 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.929596    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:30.932710 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.932689    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 13 19:28:30.932815 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.932690    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-f2vlv\""
Jul 13 19:28:30.945689 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.945666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6ljsh"]
Jul 13 19:28:30.976392 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.976307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brcz8\" (UniqueName: \"kubernetes.io/projected/53826db9-0478-4e1f-9823-a45808e81e83-kube-api-access-brcz8\") pod \"model-serving-api-54d89bd5fc-6ljsh\" (UID: \"53826db9-0478-4e1f-9823-a45808e81e83\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:30.976509 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:30.976438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53826db9-0478-4e1f-9823-a45808e81e83-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ljsh\" (UID: \"53826db9-0478-4e1f-9823-a45808e81e83\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:31.004511 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.004487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:31.077482 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.077450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53826db9-0478-4e1f-9823-a45808e81e83-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ljsh\" (UID: \"53826db9-0478-4e1f-9823-a45808e81e83\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:31.077702 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.077524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brcz8\" (UniqueName: \"kubernetes.io/projected/53826db9-0478-4e1f-9823-a45808e81e83-kube-api-access-brcz8\") pod \"model-serving-api-54d89bd5fc-6ljsh\" (UID: \"53826db9-0478-4e1f-9823-a45808e81e83\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:31.080597 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.080572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53826db9-0478-4e1f-9823-a45808e81e83-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ljsh\" (UID: \"53826db9-0478-4e1f-9823-a45808e81e83\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:31.087435 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.087407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brcz8\" (UniqueName: \"kubernetes.io/projected/53826db9-0478-4e1f-9823-a45808e81e83-kube-api-access-brcz8\") pod \"model-serving-api-54d89bd5fc-6ljsh\" (UID: \"53826db9-0478-4e1f-9823-a45808e81e83\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:31.130841 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.130755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:31.242714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.242682    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:31.413426 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.413352    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6ljsh"]
Jul 13 19:28:31.417116 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:28:31.417093    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53826db9_0478_4e1f_9823_a45808e81e83.slice/crio-c12a59443357282d84b8ec854e3676bacce67d72b46dade3caadd9562b02c0ce WatchSource:0}: Error finding container c12a59443357282d84b8ec854e3676bacce67d72b46dade3caadd9562b02c0ce: Status 404 returned error can't find the container with id c12a59443357282d84b8ec854e3676bacce67d72b46dade3caadd9562b02c0ce
Jul 13 19:28:31.456931 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.456905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh" event={"ID":"53826db9-0478-4e1f-9823-a45808e81e83","Type":"ContainerStarted","Data":"c12a59443357282d84b8ec854e3676bacce67d72b46dade3caadd9562b02c0ce"}
Jul 13 19:28:31.600882 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.600850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:31.746890 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:31.746791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:33.032254 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:33.032218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:34.468932 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:34.468888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s" event={"ID":"c3dea633-36df-46bf-a96c-55e0a58f7567","Type":"ContainerStarted","Data":"7f91faec6e94ac044e985a13d95bbf2bf5998b108f732b5236fb2ecabfc110b8"}
Jul 13 19:28:34.470319 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:34.470293    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh" event={"ID":"53826db9-0478-4e1f-9823-a45808e81e83","Type":"ContainerStarted","Data":"1243b2eb23e8845720e878fa57c3a7896dd0b4a49a77103ad3575038066ec93b"}
Jul 13 19:28:34.470424 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:34.470340    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:34.486227 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:34.486181    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55f6586c87-mxm6s" podStartSLOduration=2.218933173 podStartE2EDuration="5.486167724s" totalImagesPullingTime="3.267234551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:34.485666556 +0000 UTC m=+410.900942739" watchObservedRunningTime="2026-07-13 19:28:34.486167724 +0000 UTC m=+410.901443909"
Jul 13 19:28:34.504271 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:34.504221    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh" podStartSLOduration=2.385996917 podStartE2EDuration="4.504207321s" totalImagesPullingTime="2.118210404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:34.503183404 +0000 UTC m=+410.918459580" watchObservedRunningTime="2026-07-13 19:28:34.504207321 +0000 UTC m=+410.919483502"
Jul 13 19:28:38.305269 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.305239    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"]
Jul 13 19:28:38.308199 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.308183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.310990 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.310968    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 13 19:28:38.311095 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.311045    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-d2fvc\""
Jul 13 19:28:38.311525 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.311510    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 13 19:28:38.311585 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.311565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 13 19:28:38.314268 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.314233    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 13 19:28:38.329813 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs6t2\" (UniqueName: \"kubernetes.io/projected/ac23756b-d61c-4d52-aaa8-42abeb01e095-kube-api-access-qs6t2\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.329896 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac23756b-d61c-4d52-aaa8-42abeb01e095-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.329896 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.329896 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.330012 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.330012 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.330012 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.329990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.355118 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.355097    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"]
Jul 13 19:28:38.430337 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.430448 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.430448 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.430448 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs6t2\" (UniqueName: \"kubernetes.io/projected/ac23756b-d61c-4d52-aaa8-42abeb01e095-kube-api-access-qs6t2\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.430448 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac23756b-d61c-4d52-aaa8-42abeb01e095-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.430448 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.430738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.430453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.431326 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.431302    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.432860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.432835    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.432860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.432853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac23756b-d61c-4d52-aaa8-42abeb01e095-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.432972 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.432935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.433007 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.432995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ac23756b-d61c-4d52-aaa8-42abeb01e095-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.440694 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.440671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac23756b-d61c-4d52-aaa8-42abeb01e095-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.442306 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.442285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs6t2\" (UniqueName: \"kubernetes.io/projected/ac23756b-d61c-4d52-aaa8-42abeb01e095-kube-api-access-qs6t2\") pod \"istiod-openshift-gateway-94bb8fbfd-w2c2q\" (UID: \"ac23756b-d61c-4d52-aaa8-42abeb01e095\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.584906 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.584875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:38.616588 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.616565    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:38.685681 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.685651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:38.757346 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.757324    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"]
Jul 13 19:28:38.759617 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:28:38.759578    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac23756b_d61c_4d52_aaa8_42abeb01e095.slice/crio-e8a6b5403eb5c462aacf0c308d513d53b8af18758df14391e154e92ab2fa1997 WatchSource:0}: Error finding container e8a6b5403eb5c462aacf0c308d513d53b8af18758df14391e154e92ab2fa1997: Status 404 returned error can't find the container with id e8a6b5403eb5c462aacf0c308d513d53b8af18758df14391e154e92ab2fa1997
Jul 13 19:28:38.834806 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:38.834775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:39.034738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:39.034646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:39.320597 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:39.320524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:39.488468 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:39.488433    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q" event={"ID":"ac23756b-d61c-4d52-aaa8-42abeb01e095","Type":"ContainerStarted","Data":"e8a6b5403eb5c462aacf0c308d513d53b8af18758df14391e154e92ab2fa1997"}
Jul 13 19:28:42.083920 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.083881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:42.242161 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.242132    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"]
Jul 13 19:28:42.252359 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.252335    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.256774 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.256744    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 13 19:28:42.257223 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.257204    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8bl9s\""
Jul 13 19:28:42.279488 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.279465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:42.297880 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.297856    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"]
Jul 13 19:28:42.358986 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.358889    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8rpl\" (UniqueName: \"kubernetes.io/projected/1845848f-2b33-4144-9187-356b9905ec22-kube-api-access-x8rpl\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.358986 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.358962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.367410 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.367378    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:28:42.367494 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.367435    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:28:42.460385 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.460351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8rpl\" (UniqueName: \"kubernetes.io/projected/1845848f-2b33-4144-9187-356b9905ec22-kube-api-access-x8rpl\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.460511 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.460404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.460575 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:28:42.460560    2562 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 13 19:28:42.460629 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:28:42.460619    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert podName:1845848f-2b33-4144-9187-356b9905ec22 nodeName:}" failed. No retries permitted until 2026-07-13 19:28:42.960599849 +0000 UTC m=+419.375876023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert") pod "kserve-controller-manager-d6f4b65b8-xn9bx" (UID: "1845848f-2b33-4144-9187-356b9905ec22") : secret "kserve-webhook-server-cert" not found
Jul 13 19:28:42.475516 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.475491    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8rpl\" (UniqueName: \"kubernetes.io/projected/1845848f-2b33-4144-9187-356b9905ec22-kube-api-access-x8rpl\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.500577 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.500526    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q" event={"ID":"ac23756b-d61c-4d52-aaa8-42abeb01e095","Type":"ContainerStarted","Data":"ed63f229ae9c64281bed171227d5bb2b3c93bbafa968490cddc0012cd6494a5a"}
Jul 13 19:28:42.500704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.500675    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:42.573488 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.573444    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q" podStartSLOduration=0.96772442 podStartE2EDuration="4.573424814s" totalImagesPullingTime="3.605700394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:42.571115062 +0000 UTC m=+418.986391243" watchObservedRunningTime="2026-07-13 19:28:42.573424814 +0000 UTC m=+418.988700996"
Jul 13 19:28:42.681499 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.681436    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:42.963913 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:42.963828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:42.964061 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:28:42.963942    2562 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 13 19:28:42.964061 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:28:42.963994    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert podName:1845848f-2b33-4144-9187-356b9905ec22 nodeName:}" failed. No retries permitted until 2026-07-13 19:28:43.963981554 +0000 UTC m=+420.379257713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert") pod "kserve-controller-manager-d6f4b65b8-xn9bx" (UID: "1845848f-2b33-4144-9187-356b9905ec22") : secret "kserve-webhook-server-cert" not found
Jul 13 19:28:43.393608 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.393573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:43.505604 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.505575    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w2c2q"
Jul 13 19:28:43.508297 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.508278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:43.747597 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.747494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:43.927478 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.927449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:28:43.972899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.972874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:43.975240 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:43.975219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1845848f-2b33-4144-9187-356b9905ec22-cert\") pod \"kserve-controller-manager-d6f4b65b8-xn9bx\" (UID: \"1845848f-2b33-4144-9187-356b9905ec22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:44.067984 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:44.067959    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8bl9s\""
Jul 13 19:28:44.076099 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:44.076073    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:44.228708 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:44.228687    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"]
Jul 13 19:28:44.231193 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:28:44.231162    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1845848f_2b33_4144_9187_356b9905ec22.slice/crio-9ec9736cb3f14d1af6742370a13628c9e836c338fb7c679117c43d25843fabea WatchSource:0}: Error finding container 9ec9736cb3f14d1af6742370a13628c9e836c338fb7c679117c43d25843fabea: Status 404 returned error can't find the container with id 9ec9736cb3f14d1af6742370a13628c9e836c338fb7c679117c43d25843fabea
Jul 13 19:28:44.508769 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:44.508674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx" event={"ID":"1845848f-2b33-4144-9187-356b9905ec22","Type":"ContainerStarted","Data":"9ec9736cb3f14d1af6742370a13628c9e836c338fb7c679117c43d25843fabea"}
Jul 13 19:28:45.478445 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:45.478415    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6ljsh"
Jul 13 19:28:47.522788 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:47.522748    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx" event={"ID":"1845848f-2b33-4144-9187-356b9905ec22","Type":"ContainerStarted","Data":"29ebf3167295b34b527087d4d4f0b1401b111a72565920f51cf7dd8886f89c35"}
Jul 13 19:28:47.523178 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:47.522864    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:28:47.541486 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:28:47.541438    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx" podStartSLOduration=2.7598581319999997 podStartE2EDuration="5.541420983s" totalImagesPullingTime="2.781562851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:47.541054309 +0000 UTC m=+423.956330492" watchObservedRunningTime="2026-07-13 19:28:47.541420983 +0000 UTC m=+423.956697164"
Jul 13 19:29:18.531337 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:18.531254    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xn9bx"
Jul 13 19:29:30.722309 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.722275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:30.819187 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.819154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:30.879265 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.879238    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4rm52"]
Jul 13 19:29:30.887596 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.887575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:30.890644 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.890622    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 19:29:30.892055 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.892038    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 19:29:30.892241 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.891989    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9d4nx\""
Jul 13 19:29:30.893548 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.893509    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4rm52"]
Jul 13 19:29:30.935644 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:30.935618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:31.024225 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.024147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n4wm\" (UniqueName: \"kubernetes.io/projected/af93d376-e5af-484a-afcd-888a7f905cee-kube-api-access-2n4wm\") pod \"kuadrant-operator-catalog-4rm52\" (UID: \"af93d376-e5af-484a-afcd-888a7f905cee\") " pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:31.124769 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.124727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n4wm\" (UniqueName: \"kubernetes.io/projected/af93d376-e5af-484a-afcd-888a7f905cee-kube-api-access-2n4wm\") pod \"kuadrant-operator-catalog-4rm52\" (UID: \"af93d376-e5af-484a-afcd-888a7f905cee\") " pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:31.134111 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.134075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n4wm\" (UniqueName: \"kubernetes.io/projected/af93d376-e5af-484a-afcd-888a7f905cee-kube-api-access-2n4wm\") pod \"kuadrant-operator-catalog-4rm52\" (UID: \"af93d376-e5af-484a-afcd-888a7f905cee\") " pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:31.198152 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.198117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:31.239125 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.239094    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4rm52"]
Jul 13 19:29:31.326867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.326848    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4rm52"]
Jul 13 19:29:31.329035 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:29:31.329007    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf93d376_e5af_484a_afcd_888a7f905cee.slice/crio-21e6070253fa803db2bfd6527e3445fa7f62a669403342e1d9838753123261f9 WatchSource:0}: Error finding container 21e6070253fa803db2bfd6527e3445fa7f62a669403342e1d9838753123261f9: Status 404 returned error can't find the container with id 21e6070253fa803db2bfd6527e3445fa7f62a669403342e1d9838753123261f9
Jul 13 19:29:31.451747 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.451717    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ftkvv"]
Jul 13 19:29:31.455940 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.455926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:31.464137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.464113    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ftkvv"]
Jul 13 19:29:31.627297 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.627262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bk7k\" (UniqueName: \"kubernetes.io/projected/30c9db42-b3ec-473e-bbe8-9819a02bd534-kube-api-access-4bk7k\") pod \"kuadrant-operator-catalog-ftkvv\" (UID: \"30c9db42-b3ec-473e-bbe8-9819a02bd534\") " pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:31.675188 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.675154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4rm52" event={"ID":"af93d376-e5af-484a-afcd-888a7f905cee","Type":"ContainerStarted","Data":"21e6070253fa803db2bfd6527e3445fa7f62a669403342e1d9838753123261f9"}
Jul 13 19:29:31.728222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.728196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bk7k\" (UniqueName: \"kubernetes.io/projected/30c9db42-b3ec-473e-bbe8-9819a02bd534-kube-api-access-4bk7k\") pod \"kuadrant-operator-catalog-ftkvv\" (UID: \"30c9db42-b3ec-473e-bbe8-9819a02bd534\") " pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:31.741753 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.741734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bk7k\" (UniqueName: \"kubernetes.io/projected/30c9db42-b3ec-473e-bbe8-9819a02bd534-kube-api-access-4bk7k\") pod \"kuadrant-operator-catalog-ftkvv\" (UID: \"30c9db42-b3ec-473e-bbe8-9819a02bd534\") " pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:31.765653 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.765628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:31.921162 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:31.921032    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ftkvv"]
Jul 13 19:29:31.957219 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:29:31.957189    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c9db42_b3ec_473e_bbe8_9819a02bd534.slice/crio-6809d5b9feef745d857cef337d412adbf3e29d80d3256f7d1efb7f8d26ee291c WatchSource:0}: Error finding container 6809d5b9feef745d857cef337d412adbf3e29d80d3256f7d1efb7f8d26ee291c: Status 404 returned error can't find the container with id 6809d5b9feef745d857cef337d412adbf3e29d80d3256f7d1efb7f8d26ee291c
Jul 13 19:29:32.681372 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:32.681334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv" event={"ID":"30c9db42-b3ec-473e-bbe8-9819a02bd534","Type":"ContainerStarted","Data":"6809d5b9feef745d857cef337d412adbf3e29d80d3256f7d1efb7f8d26ee291c"}
Jul 13 19:29:34.694125 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:34.694085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4rm52" event={"ID":"af93d376-e5af-484a-afcd-888a7f905cee","Type":"ContainerStarted","Data":"4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975"}
Jul 13 19:29:34.694125 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:34.694111    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4rm52" podUID="af93d376-e5af-484a-afcd-888a7f905cee" containerName="registry-server" containerID="cri-o://4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975" gracePeriod=2
Jul 13 19:29:34.695451 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:34.695424    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv" event={"ID":"30c9db42-b3ec-473e-bbe8-9819a02bd534","Type":"ContainerStarted","Data":"b334b0a885668c036be901c66b49ef46ed0434549a08a00c5a89cd0d8fdce134"}
Jul 13 19:29:34.709837 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:34.709798    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4rm52" podStartSLOduration=2.372337357 podStartE2EDuration="4.709783213s" totalImagesPullingTime="2.337445856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:29:34.708880224 +0000 UTC m=+471.124156406" watchObservedRunningTime="2026-07-13 19:29:34.709783213 +0000 UTC m=+471.125059405"
Jul 13 19:29:34.725036 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:34.724996    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv" podStartSLOduration=2.012802247 podStartE2EDuration="3.724981742s" totalImagesPullingTime="1.712179495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:29:34.724177961 +0000 UTC m=+471.139454144" watchObservedRunningTime="2026-07-13 19:29:34.724981742 +0000 UTC m=+471.140257924"
Jul 13 19:29:34.934454 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:34.934431    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:35.054824 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.054749    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n4wm\" (UniqueName: \"kubernetes.io/projected/af93d376-e5af-484a-afcd-888a7f905cee-kube-api-access-2n4wm\") pod \"af93d376-e5af-484a-afcd-888a7f905cee\" (UID: \"af93d376-e5af-484a-afcd-888a7f905cee\") "
Jul 13 19:29:35.057116 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.057091    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af93d376-e5af-484a-afcd-888a7f905cee-kube-api-access-2n4wm" (OuterVolumeSpecName: "kube-api-access-2n4wm") pod "af93d376-e5af-484a-afcd-888a7f905cee" (UID: "af93d376-e5af-484a-afcd-888a7f905cee"). InnerVolumeSpecName "kube-api-access-2n4wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:29:35.155228 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.155196    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n4wm\" (UniqueName: \"kubernetes.io/projected/af93d376-e5af-484a-afcd-888a7f905cee-kube-api-access-2n4wm\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:29:35.700322 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.700284    2562 generic.go:356] "Generic (PLEG): container finished" podID="af93d376-e5af-484a-afcd-888a7f905cee" containerID="4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975" exitCode=0
Jul 13 19:29:35.700779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.700329    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4rm52"
Jul 13 19:29:35.700779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.700364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4rm52" event={"ID":"af93d376-e5af-484a-afcd-888a7f905cee","Type":"ContainerDied","Data":"4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975"}
Jul 13 19:29:35.700779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.700410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4rm52" event={"ID":"af93d376-e5af-484a-afcd-888a7f905cee","Type":"ContainerDied","Data":"21e6070253fa803db2bfd6527e3445fa7f62a669403342e1d9838753123261f9"}
Jul 13 19:29:35.700779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.700430    2562 scope.go:117] "RemoveContainer" containerID="4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975"
Jul 13 19:29:35.712269 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.712249    2562 scope.go:117] "RemoveContainer" containerID="4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975"
Jul 13 19:29:35.712574 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:29:35.712554    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975\": container with ID starting with 4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975 not found: ID does not exist" containerID="4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975"
Jul 13 19:29:35.712633 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.712582    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975"} err="failed to get container status \"4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975\": rpc error: code = NotFound desc = could not find container \"4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975\": container with ID starting with 4c4025199430b97e99a31274584c9d33f6837c0f7f25ce2d3bd27e32fa14a975 not found: ID does not exist"
Jul 13 19:29:35.723740 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.723708    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4rm52"]
Jul 13 19:29:35.726866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:35.726846    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4rm52"]
Jul 13 19:29:36.089807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:36.089782    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af93d376-e5af-484a-afcd-888a7f905cee" path="/var/lib/kubelet/pods/af93d376-e5af-484a-afcd-888a7f905cee/volumes"
Jul 13 19:29:41.766150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:41.766111    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:41.766691 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:41.766188    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:41.788651 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:41.788619    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:41.877446 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:41.877413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:41.976542 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:41.976512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:42.748792 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:42.748766    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ftkvv"
Jul 13 19:29:42.758702 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:42.758675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:42.875521 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:42.875490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:46.764906 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:46.764878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:46.864801 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:46.864770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:57.721666 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:57.721633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:57.784486 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:57.784454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:29:57.841883 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:29:57.841857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:04.347107 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:04.347076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:08.563011 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:08.562980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:10.443062 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.443029    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"]
Jul 13 19:30:10.443401 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.443286    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af93d376-e5af-484a-afcd-888a7f905cee" containerName="registry-server"
Jul 13 19:30:10.443401 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.443297    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="af93d376-e5af-484a-afcd-888a7f905cee" containerName="registry-server"
Jul 13 19:30:10.443401 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.443377    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="af93d376-e5af-484a-afcd-888a7f905cee" containerName="registry-server"
Jul 13 19:30:10.446138 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.446122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:10.448730 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.448714    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qglb9\""
Jul 13 19:30:10.458490 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.458468    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"]
Jul 13 19:30:10.503746 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.503720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x75t\" (UniqueName: \"kubernetes.io/projected/468b9432-930f-41f9-abec-27ae33414e18-kube-api-access-7x75t\") pod \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" (UID: \"468b9432-930f-41f9-abec-27ae33414e18\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:10.604214 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.604186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x75t\" (UniqueName: \"kubernetes.io/projected/468b9432-930f-41f9-abec-27ae33414e18-kube-api-access-7x75t\") pod \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" (UID: \"468b9432-930f-41f9-abec-27ae33414e18\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:10.613231 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.613207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x75t\" (UniqueName: \"kubernetes.io/projected/468b9432-930f-41f9-abec-27ae33414e18-kube-api-access-7x75t\") pod \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" (UID: \"468b9432-930f-41f9-abec-27ae33414e18\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:10.755932 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.755864    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:10.885557 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.885477    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"]
Jul 13 19:30:10.888194 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:30:10.888168    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468b9432_930f_41f9_abec_27ae33414e18.slice/crio-e81431215aab27d4c955d2dc1e0f5283fc8808fe74f056b419dece270b8b43bd WatchSource:0}: Error finding container e81431215aab27d4c955d2dc1e0f5283fc8808fe74f056b419dece270b8b43bd: Status 404 returned error can't find the container with id e81431215aab27d4c955d2dc1e0f5283fc8808fe74f056b419dece270b8b43bd
Jul 13 19:30:10.987123 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:10.987098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:11.289821 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:11.289785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:11.833716 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:11.833686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh" event={"ID":"468b9432-930f-41f9-abec-27ae33414e18","Type":"ContainerStarted","Data":"e81431215aab27d4c955d2dc1e0f5283fc8808fe74f056b419dece270b8b43bd"}
Jul 13 19:30:12.432779 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:12.432743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:13.843301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:13.843264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh" event={"ID":"468b9432-930f-41f9-abec-27ae33414e18","Type":"ContainerStarted","Data":"0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40"}
Jul 13 19:30:13.843687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:13.843400    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:13.861725 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:13.861680    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh" podStartSLOduration=1.872278942 podStartE2EDuration="3.861665045s" totalImagesPullingTime="1.989386103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:13.860686619 +0000 UTC m=+510.275962802" watchObservedRunningTime="2026-07-13 19:30:13.861665045 +0000 UTC m=+510.276941227"
Jul 13 19:30:16.802424 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:16.802386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:18.049206 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:18.049169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:19.026507 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:19.026452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:19.424013 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:19.423975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:21.362330 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:21.362293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:21.469159 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:21.469119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:24.760624 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:24.760578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:24.831311 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:24.831280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:24.849406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:24.849383    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:27.248931 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:27.248899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:27.406365 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:27.406337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:28.145692 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:28.145658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:28.207999 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:28.207970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:32.769288 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:32.769244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:32.869017 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:32.868984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:37.596704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.596622    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"]
Jul 13 19:30:37.597093 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.596864    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh" podUID="468b9432-930f-41f9-abec-27ae33414e18" containerName="manager" containerID="cri-o://0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40" gracePeriod=2
Jul 13 19:30:37.608007 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.607984    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"]
Jul 13 19:30:37.620489 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:30:37.620455    2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"468b9432-930f-41f9-abec-27ae33414e18\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-g4xbh\": pods \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.625961 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.625933    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"]
Jul 13 19:30:37.626278 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.626263    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468b9432-930f-41f9-abec-27ae33414e18" containerName="manager"
Jul 13 19:30:37.626344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.626280    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="468b9432-930f-41f9-abec-27ae33414e18" containerName="manager"
Jul 13 19:30:37.626344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.626335    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="468b9432-930f-41f9-abec-27ae33414e18" containerName="manager"
Jul 13 19:30:37.629297 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.629279    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:37.631639 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:30:37.631604    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" is forbidden: User \"system:node:ip-10-0-137-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-89.ec2.internal' and this object" podUID="468b9432-930f-41f9-abec-27ae33414e18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.641979 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.641957    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"]
Jul 13 19:30:37.701476 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.701443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjkbs\" (UniqueName: \"kubernetes.io/projected/352a0ab9-f732-40ad-b511-b6c086aecf91-kube-api-access-hjkbs\") pod \"limitador-operator-controller-manager-859d7b78bd-5rp8m\" (UID: \"352a0ab9-f732-40ad-b511-b6c086aecf91\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:37.802007 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.801972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjkbs\" (UniqueName: \"kubernetes.io/projected/352a0ab9-f732-40ad-b511-b6c086aecf91-kube-api-access-hjkbs\") pod \"limitador-operator-controller-manager-859d7b78bd-5rp8m\" (UID: \"352a0ab9-f732-40ad-b511-b6c086aecf91\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:37.810795 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.810773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjkbs\" (UniqueName: \"kubernetes.io/projected/352a0ab9-f732-40ad-b511-b6c086aecf91-kube-api-access-hjkbs\") pod \"limitador-operator-controller-manager-859d7b78bd-5rp8m\" (UID: \"352a0ab9-f732-40ad-b511-b6c086aecf91\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:37.832927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.832909    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.835149 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:30:37.835128    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" is forbidden: User \"system:node:ip-10-0-137-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-89.ec2.internal' and this object" podUID="468b9432-930f-41f9-abec-27ae33414e18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.902871 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.902806    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x75t\" (UniqueName: \"kubernetes.io/projected/468b9432-930f-41f9-abec-27ae33414e18-kube-api-access-7x75t\") pod \"468b9432-930f-41f9-abec-27ae33414e18\" (UID: \"468b9432-930f-41f9-abec-27ae33414e18\") "
Jul 13 19:30:37.904910 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.904884    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468b9432-930f-41f9-abec-27ae33414e18-kube-api-access-7x75t" (OuterVolumeSpecName: "kube-api-access-7x75t") pod "468b9432-930f-41f9-abec-27ae33414e18" (UID: "468b9432-930f-41f9-abec-27ae33414e18"). InnerVolumeSpecName "kube-api-access-7x75t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:30:37.936500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.936473    2562 generic.go:356] "Generic (PLEG): container finished" podID="468b9432-930f-41f9-abec-27ae33414e18" containerID="0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40" exitCode=0
Jul 13 19:30:37.936621 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.936517    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.936621 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.936584    2562 scope.go:117] "RemoveContainer" containerID="0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40"
Jul 13 19:30:37.938961 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:30:37.938934    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" is forbidden: User \"system:node:ip-10-0-137-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-89.ec2.internal' and this object" podUID="468b9432-930f-41f9-abec-27ae33414e18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.944942 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.944923    2562 scope.go:117] "RemoveContainer" containerID="0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40"
Jul 13 19:30:37.945200 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:30:37.945180    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40\": container with ID starting with 0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40 not found: ID does not exist" containerID="0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40"
Jul 13 19:30:37.945272 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.945214    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40"} err="failed to get container status \"0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40\": rpc error: code = NotFound desc = could not find container \"0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40\": container with ID starting with 0eaf87d00dcdac035fb5fce22fa2d968da26139179c4db16df313b374e5e6f40 not found: ID does not exist"
Jul 13 19:30:37.946354 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:30:37.946338    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4xbh\" is forbidden: User \"system:node:ip-10-0-137-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-89.ec2.internal' and this object" podUID="468b9432-930f-41f9-abec-27ae33414e18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4xbh"
Jul 13 19:30:37.975485 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:37.975468    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:38.003671 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.003633    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x75t\" (UniqueName: \"kubernetes.io/projected/468b9432-930f-41f9-abec-27ae33414e18-kube-api-access-7x75t\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:30:38.099292 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.099260    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468b9432-930f-41f9-abec-27ae33414e18" path="/var/lib/kubelet/pods/468b9432-930f-41f9-abec-27ae33414e18/volumes"
Jul 13 19:30:38.121761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.121739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"]
Jul 13 19:30:38.124849 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:30:38.124825    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352a0ab9_f732_40ad_b511_b6c086aecf91.slice/crio-31cab698bef532d3577400dd0f4757492ac6573a536bc6e3184e2f78fa7a5e29 WatchSource:0}: Error finding container 31cab698bef532d3577400dd0f4757492ac6573a536bc6e3184e2f78fa7a5e29: Status 404 returned error can't find the container with id 31cab698bef532d3577400dd0f4757492ac6573a536bc6e3184e2f78fa7a5e29
Jul 13 19:30:38.381139 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.381101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:38.480570 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.480434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:38.943104 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.943072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m" event={"ID":"352a0ab9-f732-40ad-b511-b6c086aecf91","Type":"ContainerStarted","Data":"ea14885ac73418d26602f6469251f8d41b7a21313be3a91b888368c7cceafd30"}
Jul 13 19:30:38.943557 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.943123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m" event={"ID":"352a0ab9-f732-40ad-b511-b6c086aecf91","Type":"ContainerStarted","Data":"31cab698bef532d3577400dd0f4757492ac6573a536bc6e3184e2f78fa7a5e29"}
Jul 13 19:30:38.943557 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.943207    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:38.968898 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.968854    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m" podStartSLOduration=1.968839357 podStartE2EDuration="1.968839357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:38.967650595 +0000 UTC m=+535.382926776" watchObservedRunningTime="2026-07-13 19:30:38.968839357 +0000 UTC m=+535.384115539"
Jul 13 19:30:38.986415 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:38.986389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:39.096414 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:39.096357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:42.949931 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:42.949884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:43.053348 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:43.053293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:45.997700 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:45.997643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:46.097349 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:46.097303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:49.949947 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:49.949917    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5rp8m"
Jul 13 19:30:50.676611 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:50.676578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:50.777123 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:50.777081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:58.173332 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:58.173293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:30:58.270760 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:30:58.270723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:00.785565 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:00.785499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:00.886014 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:00.885978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:03.590137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:03.590099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:03.691867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:03.691835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:03.978056 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:03.977971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:04.080590 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:04.080557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:04.308399 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:04.308374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:06.457819 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:06.457790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:06.558225 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:06.558193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:07.607280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:07.607250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:07.703882 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:07.703855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:13.296189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:13.296159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:17.276790 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:17.276757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:17.374628 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:17.374578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:17.567558 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:17.567467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:17.626528 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:17.626501    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:23.930219 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:23.930186    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:23.933280 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:23.933264    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:23.936036 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:23.936012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 19:31:23.936791 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:23.936778    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-88br4\""
Jul 13 19:31:23.945256 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:23.945237    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:24.020728 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.020694    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:24.036646 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.036615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xcqh\" (UniqueName: \"kubernetes.io/projected/1ee7e689-12f1-4420-a1b8-da406edbcd94-kube-api-access-9xcqh\") pod \"limitador-limitador-64c94fd64f-vglkw\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.036797 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.036654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1ee7e689-12f1-4420-a1b8-da406edbcd94-config-file\") pod \"limitador-limitador-64c94fd64f-vglkw\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.137949 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.137917    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xcqh\" (UniqueName: \"kubernetes.io/projected/1ee7e689-12f1-4420-a1b8-da406edbcd94-kube-api-access-9xcqh\") pod \"limitador-limitador-64c94fd64f-vglkw\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.138122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.138009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1ee7e689-12f1-4420-a1b8-da406edbcd94-config-file\") pod \"limitador-limitador-64c94fd64f-vglkw\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.138692 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.138665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1ee7e689-12f1-4420-a1b8-da406edbcd94-config-file\") pod \"limitador-limitador-64c94fd64f-vglkw\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.147623 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.147593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xcqh\" (UniqueName: \"kubernetes.io/projected/1ee7e689-12f1-4420-a1b8-da406edbcd94-kube-api-access-9xcqh\") pod \"limitador-limitador-64c94fd64f-vglkw\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.243506 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.243422    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:24.388095 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.388071    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:24.390479 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:31:24.390454    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee7e689_12f1_4420_a1b8_da406edbcd94.slice/crio-35cc3a886f3ef76c76f07eda14edc6636be62accc925a9d024484f33ec7828b2 WatchSource:0}: Error finding container 35cc3a886f3ef76c76f07eda14edc6636be62accc925a9d024484f33ec7828b2: Status 404 returned error can't find the container with id 35cc3a886f3ef76c76f07eda14edc6636be62accc925a9d024484f33ec7828b2
Jul 13 19:31:24.676738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.676707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:24.776787 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.776760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:24.913527 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:24.913498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:25.023608 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:25.023512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:25.116410 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:25.116373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw" event={"ID":"1ee7e689-12f1-4420-a1b8-da406edbcd94","Type":"ContainerStarted","Data":"35cc3a886f3ef76c76f07eda14edc6636be62accc925a9d024484f33ec7828b2"}
Jul 13 19:31:28.131778 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:28.131741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw" event={"ID":"1ee7e689-12f1-4420-a1b8-da406edbcd94","Type":"ContainerStarted","Data":"bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3"}
Jul 13 19:31:28.132141 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:28.131877    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:28.151250 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:28.151204    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw" podStartSLOduration=2.483611478 podStartE2EDuration="5.151189857s" totalImagesPullingTime="2.667578379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:28.149843193 +0000 UTC m=+584.565119377" watchObservedRunningTime="2026-07-13 19:31:28.151189857 +0000 UTC m=+584.566466039"
Jul 13 19:31:39.136318 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:39.136290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:39.192563 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:39.192520    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:39.192709 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:39.192691    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw" podUID="1ee7e689-12f1-4420-a1b8-da406edbcd94" containerName="limitador" containerID="cri-o://bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3" gracePeriod=30
Jul 13 19:31:40.129901 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.129879    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:40.176331 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.176307    2562 generic.go:356] "Generic (PLEG): container finished" podID="1ee7e689-12f1-4420-a1b8-da406edbcd94" containerID="bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3" exitCode=0
Jul 13 19:31:40.176670 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.176353    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw"
Jul 13 19:31:40.176670 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.176376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw" event={"ID":"1ee7e689-12f1-4420-a1b8-da406edbcd94","Type":"ContainerDied","Data":"bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3"}
Jul 13 19:31:40.176670 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.176415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vglkw" event={"ID":"1ee7e689-12f1-4420-a1b8-da406edbcd94","Type":"ContainerDied","Data":"35cc3a886f3ef76c76f07eda14edc6636be62accc925a9d024484f33ec7828b2"}
Jul 13 19:31:40.176670 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.176437    2562 scope.go:117] "RemoveContainer" containerID="bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3"
Jul 13 19:31:40.184241 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.184218    2562 scope.go:117] "RemoveContainer" containerID="bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3"
Jul 13 19:31:40.184498 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:31:40.184479    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3\": container with ID starting with bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3 not found: ID does not exist" containerID="bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3"
Jul 13 19:31:40.184577 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.184507    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3"} err="failed to get container status \"bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3\": rpc error: code = NotFound desc = could not find container \"bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3\": container with ID starting with bcffafd3eee9fd397a98e812f922ad0556f234c936d4c857406c9bb44802e8d3 not found: ID does not exist"
Jul 13 19:31:40.258859 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.258796    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1ee7e689-12f1-4420-a1b8-da406edbcd94-config-file\") pod \"1ee7e689-12f1-4420-a1b8-da406edbcd94\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") "
Jul 13 19:31:40.258859 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.258833    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xcqh\" (UniqueName: \"kubernetes.io/projected/1ee7e689-12f1-4420-a1b8-da406edbcd94-kube-api-access-9xcqh\") pod \"1ee7e689-12f1-4420-a1b8-da406edbcd94\" (UID: \"1ee7e689-12f1-4420-a1b8-da406edbcd94\") "
Jul 13 19:31:40.259186 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.259161    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee7e689-12f1-4420-a1b8-da406edbcd94-config-file" (OuterVolumeSpecName: "config-file") pod "1ee7e689-12f1-4420-a1b8-da406edbcd94" (UID: "1ee7e689-12f1-4420-a1b8-da406edbcd94"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:31:40.260973 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.260954    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee7e689-12f1-4420-a1b8-da406edbcd94-kube-api-access-9xcqh" (OuterVolumeSpecName: "kube-api-access-9xcqh") pod "1ee7e689-12f1-4420-a1b8-da406edbcd94" (UID: "1ee7e689-12f1-4420-a1b8-da406edbcd94"). InnerVolumeSpecName "kube-api-access-9xcqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:31:40.359816 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.359791    2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1ee7e689-12f1-4420-a1b8-da406edbcd94-config-file\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:31:40.359816 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.359810    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xcqh\" (UniqueName: \"kubernetes.io/projected/1ee7e689-12f1-4420-a1b8-da406edbcd94-kube-api-access-9xcqh\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:31:40.502059 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.498782    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:40.504801 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:40.504776    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vglkw"]
Jul 13 19:31:42.090094 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:42.090056    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ee7e689-12f1-4420-a1b8-da406edbcd94" path="/var/lib/kubelet/pods/1ee7e689-12f1-4420-a1b8-da406edbcd94/volumes"
Jul 13 19:31:44.746806 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:44.746773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:45.021355 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.021248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:45.190422 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.190394    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-286pl"]
Jul 13 19:31:45.190735 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.190721    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ee7e689-12f1-4420-a1b8-da406edbcd94" containerName="limitador"
Jul 13 19:31:45.190783 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.190737    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ee7e689-12f1-4420-a1b8-da406edbcd94" containerName="limitador"
Jul 13 19:31:45.190816 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.190798    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ee7e689-12f1-4420-a1b8-da406edbcd94" containerName="limitador"
Jul 13 19:31:45.195114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.195086    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.198012 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.197993    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5mssv\""
Jul 13 19:31:45.198137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.198036    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 13 19:31:45.199114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.199093    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:31:45.199194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.199117    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:31:45.203120 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.203090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-286pl"]
Jul 13 19:31:45.232710 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.232684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:45.295875 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.295818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7kj2\" (UniqueName: \"kubernetes.io/projected/a2052937-bcba-4a1a-936d-bd8bb3301074-kube-api-access-h7kj2\") pod \"postgres-69fb4594cf-286pl\" (UID: \"a2052937-bcba-4a1a-936d-bd8bb3301074\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.295875 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.295859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2052937-bcba-4a1a-936d-bd8bb3301074-data\") pod \"postgres-69fb4594cf-286pl\" (UID: \"a2052937-bcba-4a1a-936d-bd8bb3301074\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.396395 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.396366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2052937-bcba-4a1a-936d-bd8bb3301074-data\") pod \"postgres-69fb4594cf-286pl\" (UID: \"a2052937-bcba-4a1a-936d-bd8bb3301074\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.396583 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.396430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7kj2\" (UniqueName: \"kubernetes.io/projected/a2052937-bcba-4a1a-936d-bd8bb3301074-kube-api-access-h7kj2\") pod \"postgres-69fb4594cf-286pl\" (UID: \"a2052937-bcba-4a1a-936d-bd8bb3301074\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.396812 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.396790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2052937-bcba-4a1a-936d-bd8bb3301074-data\") pod \"postgres-69fb4594cf-286pl\" (UID: \"a2052937-bcba-4a1a-936d-bd8bb3301074\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.404719 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.404690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7kj2\" (UniqueName: \"kubernetes.io/projected/a2052937-bcba-4a1a-936d-bd8bb3301074-kube-api-access-h7kj2\") pod \"postgres-69fb4594cf-286pl\" (UID: \"a2052937-bcba-4a1a-936d-bd8bb3301074\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.505392 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.505349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:45.630037 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:45.630014    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-286pl"]
Jul 13 19:31:45.632340 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:31:45.632317    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2052937_bcba_4a1a_936d_bd8bb3301074.slice/crio-321275aa9d0d86b0fdcf44450a8fb8c4a7af550cf0ec605797d7545595e9bbc1 WatchSource:0}: Error finding container 321275aa9d0d86b0fdcf44450a8fb8c4a7af550cf0ec605797d7545595e9bbc1: Status 404 returned error can't find the container with id 321275aa9d0d86b0fdcf44450a8fb8c4a7af550cf0ec605797d7545595e9bbc1
Jul 13 19:31:46.202899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:46.202864    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl" event={"ID":"a2052937-bcba-4a1a-936d-bd8bb3301074","Type":"ContainerStarted","Data":"321275aa9d0d86b0fdcf44450a8fb8c4a7af550cf0ec605797d7545595e9bbc1"}
Jul 13 19:31:51.227761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:51.227718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl" event={"ID":"a2052937-bcba-4a1a-936d-bd8bb3301074","Type":"ContainerStarted","Data":"97dc5aeba564279f7d14e47d70dfbe0fc116945e48203fdc321dbbe6d9c689da"}
Jul 13 19:31:51.228150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:51.227822    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:51.243876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:51.243830    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl" podStartSLOduration=1.178625836 podStartE2EDuration="6.243814862s" totalImagesPullingTime="5.065189026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:51.24221184 +0000 UTC m=+607.657488020" watchObservedRunningTime="2026-07-13 19:31:51.243814862 +0000 UTC m=+607.659091044"
Jul 13 19:31:57.260961 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:57.260928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-286pl"
Jul 13 19:31:57.584741 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:57.584712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:31:58.912150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:31:58.912102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:04.701297 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.701191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:04.725798 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.725773    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8565894bbf-chv5t"]
Jul 13 19:32:04.747913 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.747876    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8565894bbf-chv5t"]
Jul 13 19:32:04.748060 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.747967    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:04.755399 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.755377    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 13 19:32:04.755597 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.755494    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-46flg\""
Jul 13 19:32:04.755674 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.755626    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 13 19:32:04.824105 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.824073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:04.851367 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.851343    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ptg9\" (UniqueName: \"kubernetes.io/projected/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-kube-api-access-7ptg9\") pod \"maas-controller-8565894bbf-chv5t\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") " pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:04.851470 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.851407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-webhook-cert\") pod \"maas-controller-8565894bbf-chv5t\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") " pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:04.917231 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.917202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:04.927466 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.927439    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-84h9n"]
Jul 13 19:32:04.929482 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.929467    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:04.941435 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.941414    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-84h9n"]
Jul 13 19:32:04.952418 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.952361    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ptg9\" (UniqueName: \"kubernetes.io/projected/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-kube-api-access-7ptg9\") pod \"maas-controller-8565894bbf-chv5t\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") " pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:04.952497 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.952417    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-webhook-cert\") pod \"maas-controller-8565894bbf-chv5t\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") " pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:04.954784 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.954765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-webhook-cert\") pod \"maas-controller-8565894bbf-chv5t\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") " pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:04.964010 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:04.963988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ptg9\" (UniqueName: \"kubernetes.io/projected/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-kube-api-access-7ptg9\") pod \"maas-controller-8565894bbf-chv5t\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") " pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:05.052929 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.052900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-webhook-cert\") pod \"maas-controller-7d8fb89cc8-84h9n\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") " pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.053042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.052934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dv8m\" (UniqueName: \"kubernetes.io/projected/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-kube-api-access-6dv8m\") pod \"maas-controller-7d8fb89cc8-84h9n\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") " pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.058881 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.058857    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:05.153672 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.153638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-webhook-cert\") pod \"maas-controller-7d8fb89cc8-84h9n\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") " pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.153672 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.153674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dv8m\" (UniqueName: \"kubernetes.io/projected/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-kube-api-access-6dv8m\") pod \"maas-controller-7d8fb89cc8-84h9n\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") " pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.156030 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.156007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-webhook-cert\") pod \"maas-controller-7d8fb89cc8-84h9n\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") " pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.162807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.162782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dv8m\" (UniqueName: \"kubernetes.io/projected/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-kube-api-access-6dv8m\") pod \"maas-controller-7d8fb89cc8-84h9n\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") " pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.187260 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.187239    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8565894bbf-chv5t"]
Jul 13 19:32:05.189056 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:32:05.189033    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa60407c_5b3e_4b2a_8428_a8d5430f8f35.slice/crio-ef72516d4872c1414129178a0e939ba21e747e945355236531e04e84bec78824 WatchSource:0}: Error finding container ef72516d4872c1414129178a0e939ba21e747e945355236531e04e84bec78824: Status 404 returned error can't find the container with id ef72516d4872c1414129178a0e939ba21e747e945355236531e04e84bec78824
Jul 13 19:32:05.239457 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.239405    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:05.287680 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.287650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8565894bbf-chv5t" event={"ID":"fa60407c-5b3e-4b2a-8428-a8d5430f8f35","Type":"ContainerStarted","Data":"ef72516d4872c1414129178a0e939ba21e747e945355236531e04e84bec78824"}
Jul 13 19:32:05.370026 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:05.369992    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-84h9n"]
Jul 13 19:32:05.370921 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:32:05.370889    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5d483d_c6dc_4ef1_9c27_4123b00600a8.slice/crio-a7782b239426e9120be1294105da71f8a7f16304f5c12da1d92db8d4399f5b71 WatchSource:0}: Error finding container a7782b239426e9120be1294105da71f8a7f16304f5c12da1d92db8d4399f5b71: Status 404 returned error can't find the container with id a7782b239426e9120be1294105da71f8a7f16304f5c12da1d92db8d4399f5b71
Jul 13 19:32:06.294936 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:06.294847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n" event={"ID":"cd5d483d-c6dc-4ef1-9c27-4123b00600a8","Type":"ContainerStarted","Data":"a7782b239426e9120be1294105da71f8a7f16304f5c12da1d92db8d4399f5b71"}
Jul 13 19:32:07.687486 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:07.687456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:07.788178 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:07.788103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:08.305063 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:08.305031    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8565894bbf-chv5t" event={"ID":"fa60407c-5b3e-4b2a-8428-a8d5430f8f35","Type":"ContainerStarted","Data":"5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877"}
Jul 13 19:32:08.305274 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:08.305115    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:08.306587 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:08.306565    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n" event={"ID":"cd5d483d-c6dc-4ef1-9c27-4123b00600a8","Type":"ContainerStarted","Data":"be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9"}
Jul 13 19:32:08.306713 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:08.306700    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:08.333395 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:08.333350    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8565894bbf-chv5t" podStartSLOduration=2.040939647 podStartE2EDuration="4.333335459s" totalImagesPullingTime="2.292395812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:32:08.332484903 +0000 UTC m=+624.747761085" watchObservedRunningTime="2026-07-13 19:32:08.333335459 +0000 UTC m=+624.748611641"
Jul 13 19:32:08.358927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:08.358886    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n" podStartSLOduration=2.2522454290000002 podStartE2EDuration="4.358871973s" totalImagesPullingTime="2.106626544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:32:08.358628734 +0000 UTC m=+624.773904915" watchObservedRunningTime="2026-07-13 19:32:08.358871973 +0000 UTC m=+624.774148154"
Jul 13 19:32:19.312999 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.312966    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:32:19.313467 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.313051    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:19.382527 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.382491    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8565894bbf-chv5t"]
Jul 13 19:32:19.382746 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.382711    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8565894bbf-chv5t" podUID="fa60407c-5b3e-4b2a-8428-a8d5430f8f35" containerName="manager" containerID="cri-o://5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877" gracePeriod=30
Jul 13 19:32:19.623172 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.623150    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:19.770262 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.770230    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-webhook-cert\") pod \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") "
Jul 13 19:32:19.770454 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.770343    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ptg9\" (UniqueName: \"kubernetes.io/projected/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-kube-api-access-7ptg9\") pod \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\" (UID: \"fa60407c-5b3e-4b2a-8428-a8d5430f8f35\") "
Jul 13 19:32:19.772525 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.772494    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fa60407c-5b3e-4b2a-8428-a8d5430f8f35" (UID: "fa60407c-5b3e-4b2a-8428-a8d5430f8f35"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:32:19.772653 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.772626    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-kube-api-access-7ptg9" (OuterVolumeSpecName: "kube-api-access-7ptg9") pod "fa60407c-5b3e-4b2a-8428-a8d5430f8f35" (UID: "fa60407c-5b3e-4b2a-8428-a8d5430f8f35"). InnerVolumeSpecName "kube-api-access-7ptg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:32:19.872023 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.871941    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ptg9\" (UniqueName: \"kubernetes.io/projected/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-kube-api-access-7ptg9\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:32:19.872023 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:19.871968    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa60407c-5b3e-4b2a-8428-a8d5430f8f35-webhook-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:32:20.355676 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.355648    2562 generic.go:356] "Generic (PLEG): container finished" podID="fa60407c-5b3e-4b2a-8428-a8d5430f8f35" containerID="5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877" exitCode=0
Jul 13 19:32:20.356054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.355694    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8565894bbf-chv5t"
Jul 13 19:32:20.356054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.355727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8565894bbf-chv5t" event={"ID":"fa60407c-5b3e-4b2a-8428-a8d5430f8f35","Type":"ContainerDied","Data":"5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877"}
Jul 13 19:32:20.356054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.355775    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8565894bbf-chv5t" event={"ID":"fa60407c-5b3e-4b2a-8428-a8d5430f8f35","Type":"ContainerDied","Data":"ef72516d4872c1414129178a0e939ba21e747e945355236531e04e84bec78824"}
Jul 13 19:32:20.356054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.355808    2562 scope.go:117] "RemoveContainer" containerID="5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877"
Jul 13 19:32:20.365970 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.365951    2562 scope.go:117] "RemoveContainer" containerID="5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877"
Jul 13 19:32:20.366400 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:32:20.366377    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877\": container with ID starting with 5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877 not found: ID does not exist" containerID="5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877"
Jul 13 19:32:20.366453 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.366411    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877"} err="failed to get container status \"5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877\": rpc error: code = NotFound desc = could not find container \"5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877\": container with ID starting with 5592cdfcab59f5241e6b6d2962d06d9734b92b976fb564034d0ca98fc675c877 not found: ID does not exist"
Jul 13 19:32:20.374609 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.374588    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8565894bbf-chv5t"]
Jul 13 19:32:20.378675 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:20.378656    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8565894bbf-chv5t"]
Jul 13 19:32:22.090615 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:22.090577    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa60407c-5b3e-4b2a-8428-a8d5430f8f35" path="/var/lib/kubelet/pods/fa60407c-5b3e-4b2a-8428-a8d5430f8f35/volumes"
Jul 13 19:32:24.735978 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:24.735946    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:37.810130 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:37.810098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:37.906164 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:37.906127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:38.324277 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.324241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:38.380780 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.380748    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8bd4df569-2r472"]
Jul 13 19:32:38.381028 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.381017    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa60407c-5b3e-4b2a-8428-a8d5430f8f35" containerName="manager"
Jul 13 19:32:38.381074 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.381029    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa60407c-5b3e-4b2a-8428-a8d5430f8f35" containerName="manager"
Jul 13 19:32:38.381134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.381090    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa60407c-5b3e-4b2a-8428-a8d5430f8f35" containerName="manager"
Jul 13 19:32:38.382872 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.382857    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.385853 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.385835    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 13 19:32:38.397223 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.397200    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8bd4df569-2r472"]
Jul 13 19:32:38.400403 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.400379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58j8j\" (UniqueName: \"kubernetes.io/projected/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-kube-api-access-58j8j\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.400504 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.400426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-maas-api-tls\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.400592 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.400515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-openshift-service-ca\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.501377 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.501332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58j8j\" (UniqueName: \"kubernetes.io/projected/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-kube-api-access-58j8j\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.501619 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.501599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-maas-api-tls\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.501724 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.501663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-openshift-service-ca\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.502225 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.502208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-openshift-service-ca\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.504106 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.504086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-maas-api-tls\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.509886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.509861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58j8j\" (UniqueName: \"kubernetes.io/projected/4796d3b0-9beb-4383-b6f3-3dae3e3a9015-kube-api-access-58j8j\") pod \"maas-api-8bd4df569-2r472\" (UID: \"4796d3b0-9beb-4383-b6f3-3dae3e3a9015\") " pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.557509 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.557483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:38.692108 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.692031    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:38.822432 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.822403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8bd4df569-2r472"]
Jul 13 19:32:38.900023 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:38.899990    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:39.014927 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.014855    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"]
Jul 13 19:32:39.017487 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.017471    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.020546 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.020512    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g89pk\""
Jul 13 19:32:39.021958 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.021941    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 13 19:32:39.036806 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.036781    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"]
Jul 13 19:32:39.038116 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.038095    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-x79dq"]
Jul 13 19:32:39.039950 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.039934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.050358 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.050340    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-x79dq"]
Jul 13 19:32:39.105286 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.105265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hljn\" (UniqueName: \"kubernetes.io/projected/b4419eee-d4cb-42b0-8a10-1b90b63ccf3a-kube-api-access-5hljn\") pod \"payload-pre-processing-74584bb9f8-mp6wh\" (UID: \"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.105419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.105304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb32bdc3-2727-4d0c-b1d8-be866aa42947-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-x79dq\" (UID: \"bb32bdc3-2727-4d0c-b1d8-be866aa42947\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.105419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.105340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlrrm\" (UniqueName: \"kubernetes.io/projected/bb32bdc3-2727-4d0c-b1d8-be866aa42947-kube-api-access-wlrrm\") pod \"payload-processing-f9b8d9b9b-x79dq\" (UID: \"bb32bdc3-2727-4d0c-b1d8-be866aa42947\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.105419 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.105355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4419eee-d4cb-42b0-8a10-1b90b63ccf3a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-mp6wh\" (UID: \"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.206176 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.206148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb32bdc3-2727-4d0c-b1d8-be866aa42947-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-x79dq\" (UID: \"bb32bdc3-2727-4d0c-b1d8-be866aa42947\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.206365 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.206201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlrrm\" (UniqueName: \"kubernetes.io/projected/bb32bdc3-2727-4d0c-b1d8-be866aa42947-kube-api-access-wlrrm\") pod \"payload-processing-f9b8d9b9b-x79dq\" (UID: \"bb32bdc3-2727-4d0c-b1d8-be866aa42947\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.206365 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.206348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4419eee-d4cb-42b0-8a10-1b90b63ccf3a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-mp6wh\" (UID: \"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.206482 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.206430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hljn\" (UniqueName: \"kubernetes.io/projected/b4419eee-d4cb-42b0-8a10-1b90b63ccf3a-kube-api-access-5hljn\") pod \"payload-pre-processing-74584bb9f8-mp6wh\" (UID: \"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.206873 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.206856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb32bdc3-2727-4d0c-b1d8-be866aa42947-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-x79dq\" (UID: \"bb32bdc3-2727-4d0c-b1d8-be866aa42947\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.207094 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.207078    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4419eee-d4cb-42b0-8a10-1b90b63ccf3a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-mp6wh\" (UID: \"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.214806 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.214776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hljn\" (UniqueName: \"kubernetes.io/projected/b4419eee-d4cb-42b0-8a10-1b90b63ccf3a-kube-api-access-5hljn\") pod \"payload-pre-processing-74584bb9f8-mp6wh\" (UID: \"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.214921 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.214907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlrrm\" (UniqueName: \"kubernetes.io/projected/bb32bdc3-2727-4d0c-b1d8-be866aa42947-kube-api-access-wlrrm\") pod \"payload-processing-f9b8d9b9b-x79dq\" (UID: \"bb32bdc3-2727-4d0c-b1d8-be866aa42947\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.273085 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.273023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:39.327922 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.327897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:39.349180 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.349145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:39.430818 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.430788    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472" event={"ID":"4796d3b0-9beb-4383-b6f3-3dae3e3a9015","Type":"ContainerStarted","Data":"c43a28005637a000e69fbe8cb1bdd6290ced383bbf9eb29a20ceed9d61a099bd"}
Jul 13 19:32:39.468496 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.468471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"]
Jul 13 19:32:39.470646 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:32:39.470607    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4419eee_d4cb_42b0_8a10_1b90b63ccf3a.slice/crio-5ae9a1fad24bc78508394efe3e1bdcc9bb90ea197eb75a825cb056acfbc56999 WatchSource:0}: Error finding container 5ae9a1fad24bc78508394efe3e1bdcc9bb90ea197eb75a825cb056acfbc56999: Status 404 returned error can't find the container with id 5ae9a1fad24bc78508394efe3e1bdcc9bb90ea197eb75a825cb056acfbc56999
Jul 13 19:32:39.491684 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:39.491664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-x79dq"]
Jul 13 19:32:39.493339 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:32:39.493306    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb32bdc3_2727_4d0c_b1d8_be866aa42947.slice/crio-3bc68cee7f11c75ac6485c8727717b34c091fd139622c603aea00074bdc19a98 WatchSource:0}: Error finding container 3bc68cee7f11c75ac6485c8727717b34c091fd139622c603aea00074bdc19a98: Status 404 returned error can't find the container with id 3bc68cee7f11c75ac6485c8727717b34c091fd139622c603aea00074bdc19a98
Jul 13 19:32:40.438642 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:40.438578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq" event={"ID":"bb32bdc3-2727-4d0c-b1d8-be866aa42947","Type":"ContainerStarted","Data":"3bc68cee7f11c75ac6485c8727717b34c091fd139622c603aea00074bdc19a98"}
Jul 13 19:32:40.440573 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:40.440517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh" event={"ID":"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a","Type":"ContainerStarted","Data":"5ae9a1fad24bc78508394efe3e1bdcc9bb90ea197eb75a825cb056acfbc56999"}
Jul 13 19:32:41.448054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.447951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq" event={"ID":"bb32bdc3-2727-4d0c-b1d8-be866aa42947","Type":"ContainerStarted","Data":"d33f041c85c0ea3896e07588f92b755c27161659d9f91e5042f9941ecd158024"}
Jul 13 19:32:41.448486 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.448137    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:41.450093 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.450049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh" event={"ID":"b4419eee-d4cb-42b0-8a10-1b90b63ccf3a","Type":"ContainerStarted","Data":"9b552239c22978e56c14dde89c1464fd3470a1d8875b484866f4b400913023c3"}
Jul 13 19:32:41.450373 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.450342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:41.452312 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.452290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472" event={"ID":"4796d3b0-9beb-4383-b6f3-3dae3e3a9015","Type":"ContainerStarted","Data":"fd55e7445e7617237991861a8a91cb50163852aec694d46e9f8d5c652ec4a1ad"}
Jul 13 19:32:41.452483 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.452465    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:41.464212 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.463959    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq" podStartSLOduration=0.811172207 podStartE2EDuration="2.463942781s" totalImagesPullingTime="1.652770574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:32:41.463584751 +0000 UTC m=+657.878860933" watchObservedRunningTime="2026-07-13 19:32:41.463942781 +0000 UTC m=+657.879218963"
Jul 13 19:32:41.479648 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.479607    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472" podStartSLOduration=1.162333555 podStartE2EDuration="3.479594114s" totalImagesPullingTime="2.317260559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:32:41.47870604 +0000 UTC m=+657.893982222" watchObservedRunningTime="2026-07-13 19:32:41.479594114 +0000 UTC m=+657.894870294"
Jul 13 19:32:41.494556 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:41.494500    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh" podStartSLOduration=1.821505565 podStartE2EDuration="3.494487777s" totalImagesPullingTime="1.672982212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:32:41.493698819 +0000 UTC m=+657.908975005" watchObservedRunningTime="2026-07-13 19:32:41.494487777 +0000 UTC m=+657.909763959"
Jul 13 19:32:47.465564 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:47.462074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8bd4df569-2r472"
Jul 13 19:32:48.472127 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:48.472094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:48.569786 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:48.569757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:52.457880 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:52.457853    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-x79dq"
Jul 13 19:32:52.458462 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:52.457959    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mp6wh"
Jul 13 19:32:58.450335 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:58.450293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:32:58.505082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:32:58.505058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:06.708222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.708192    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"]
Jul 13 19:33:06.713239 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.713222    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.716250 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.716232    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 19:33:06.716903 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.716889    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4wg86\""
Jul 13 19:33:06.717128 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.717115    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 13 19:33:06.717178 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.717130    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 19:33:06.721950 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.721930    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"]
Jul 13 19:33:06.740728 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.740700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:06.816099 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.816076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.816209 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.816118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.816209 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.816136    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.816294 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.816207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/200d9b7a-7aa2-45fe-8446-ed5ced9da545-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.816294 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.816249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkd4\" (UniqueName: \"kubernetes.io/projected/200d9b7a-7aa2-45fe-8446-ed5ced9da545-kube-api-access-7qkd4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.816294 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.816272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917248 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/200d9b7a-7aa2-45fe-8446-ed5ced9da545-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917248 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkd4\" (UniqueName: \"kubernetes.io/projected/200d9b7a-7aa2-45fe-8446-ed5ced9da545-kube-api-access-7qkd4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917426 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917613 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.917734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.917718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.919343 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.919325    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/200d9b7a-7aa2-45fe-8446-ed5ced9da545-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.919602 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.919585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/200d9b7a-7aa2-45fe-8446-ed5ced9da545-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:06.925756 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:06.925731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkd4\" (UniqueName: \"kubernetes.io/projected/200d9b7a-7aa2-45fe-8446-ed5ced9da545-kube-api-access-7qkd4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb\" (UID: \"200d9b7a-7aa2-45fe-8446-ed5ced9da545\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:07.023647 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:07.023575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:07.151376 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:07.151349    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"]
Jul 13 19:33:07.154671 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:33:07.154642    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200d9b7a_7aa2_45fe_8446_ed5ced9da545.slice/crio-eb29f564b7922f99944ba91556c4acfb40ae65052e95de6994c2aea546a5a04d WatchSource:0}: Error finding container eb29f564b7922f99944ba91556c4acfb40ae65052e95de6994c2aea546a5a04d: Status 404 returned error can't find the container with id eb29f564b7922f99944ba91556c4acfb40ae65052e95de6994c2aea546a5a04d
Jul 13 19:33:07.155903 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:07.155885    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:33:07.558286 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:07.558245    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb" event={"ID":"200d9b7a-7aa2-45fe-8446-ed5ced9da545","Type":"ContainerStarted","Data":"eb29f564b7922f99944ba91556c4acfb40ae65052e95de6994c2aea546a5a04d"}
Jul 13 19:33:08.564340 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:08.564307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb" event={"ID":"200d9b7a-7aa2-45fe-8446-ed5ced9da545","Type":"ContainerStarted","Data":"c5dbd49cb2fead6ff066c08df14e9465d384e9b7795e2cc2dfda427ed38f42c2"}
Jul 13 19:33:08.583345 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:08.583051    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb" podStartSLOduration=1.420048039 podStartE2EDuration="2.58303214s" totalImagesPullingTime="1.162984101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:33:08.582653007 +0000 UTC m=+684.997929187" watchObservedRunningTime="2026-07-13 19:33:08.58303214 +0000 UTC m=+684.998308322"
Jul 13 19:33:13.703288 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.703256    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"]
Jul 13 19:33:13.707964 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.707935    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.710743 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.710726    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 13 19:33:13.714928 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.714869    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"]
Jul 13 19:33:13.750266 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.750239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:13.769434 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.769411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b5d145-3928-4a94-8985-488ecb102253-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.769510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.769446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.769510 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.769469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.769625 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.769510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.769625 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.769568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.769625 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.769588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t77w\" (UniqueName: \"kubernetes.io/projected/43b5d145-3928-4a94-8985-488ecb102253-kube-api-access-2t77w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870221    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t77w\" (UniqueName: \"kubernetes.io/projected/43b5d145-3928-4a94-8985-488ecb102253-kube-api-access-2t77w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b5d145-3928-4a94-8985-488ecb102253-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870716 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870777 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.870816 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.870777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.872599 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.872578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b5d145-3928-4a94-8985-488ecb102253-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.872975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.872957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b5d145-3928-4a94-8985-488ecb102253-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:13.879164 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:13.879139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t77w\" (UniqueName: \"kubernetes.io/projected/43b5d145-3928-4a94-8985-488ecb102253-kube-api-access-2t77w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n726q\" (UID: \"43b5d145-3928-4a94-8985-488ecb102253\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:14.018899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:14.018823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:14.152755 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:14.152664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"]
Jul 13 19:33:14.157681 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:33:14.157642    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b5d145_3928_4a94_8985_488ecb102253.slice/crio-cebff88012dbeef9d33a5f8462bd1089ecc2166e8008efc4bb6641666f84e921 WatchSource:0}: Error finding container cebff88012dbeef9d33a5f8462bd1089ecc2166e8008efc4bb6641666f84e921: Status 404 returned error can't find the container with id cebff88012dbeef9d33a5f8462bd1089ecc2166e8008efc4bb6641666f84e921
Jul 13 19:33:14.590515 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:14.590424    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q" event={"ID":"43b5d145-3928-4a94-8985-488ecb102253","Type":"ContainerStarted","Data":"5ed0da0111e388e1fbc3562bae03dc2bc5a7bae503cf4bcddf431a19d32b48e5"}
Jul 13 19:33:14.590515 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:14.590493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q" event={"ID":"43b5d145-3928-4a94-8985-488ecb102253","Type":"ContainerStarted","Data":"cebff88012dbeef9d33a5f8462bd1089ecc2166e8008efc4bb6641666f84e921"}
Jul 13 19:33:14.609082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:14.609032    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q" podStartSLOduration=1.429242344 podStartE2EDuration="1.60901393s" totalImagesPullingTime="179.771586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:33:14.608616635 +0000 UTC m=+691.023892816" watchObservedRunningTime="2026-07-13 19:33:14.60901393 +0000 UTC m=+691.024290113"
Jul 13 19:33:17.024441 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:17.024411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:17.024853 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:17.024474    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:17.036691 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:17.036671    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:17.613237 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:17.613206    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb"
Jul 13 19:33:21.616404 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:21.616371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:21.994649 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:21.994575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:24.019926 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:24.019889    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:24.020456 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:24.019956    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:24.032621 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:24.032599    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:24.643780 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:24.643745    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n726q"
Jul 13 19:33:28.349794 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:28.349764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:44.182133 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:44.182062    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:44.247293 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:44.247263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:53.011137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.011102    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"]
Jul 13 19:33:53.016789 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.016771    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.019642 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.019618    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 13 19:33:53.025058 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.025035    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"]
Jul 13 19:33:53.053107 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.053080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:33:53.095987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.095955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.096123 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.095999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npsmz\" (UniqueName: \"kubernetes.io/projected/f5aa1169-7397-4d14-835f-1daf8ff417d5-kube-api-access-npsmz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.096123 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.096027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.096123 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.096085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.096437 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.096142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5aa1169-7397-4d14-835f-1daf8ff417d5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.096437 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.096177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.196864 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.196835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npsmz\" (UniqueName: \"kubernetes.io/projected/f5aa1169-7397-4d14-835f-1daf8ff417d5-kube-api-access-npsmz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.196864 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.196866    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.196894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.196920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5aa1169-7397-4d14-835f-1daf8ff417d5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.196946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.197015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.197292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.197323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.197433 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.197391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.199282 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.199260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5aa1169-7397-4d14-835f-1daf8ff417d5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.199644 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.199622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5aa1169-7397-4d14-835f-1daf8ff417d5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.205350 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.205328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npsmz\" (UniqueName: \"kubernetes.io/projected/f5aa1169-7397-4d14-835f-1daf8ff417d5-kube-api-access-npsmz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nbpnt\" (UID: \"f5aa1169-7397-4d14-835f-1daf8ff417d5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.328107 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.328077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:33:53.459416 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.459393    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"]
Jul 13 19:33:53.461523 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:33:53.461492    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5aa1169_7397_4d14_835f_1daf8ff417d5.slice/crio-5b0e0004f563cd61bab16d26f7f72d4d2dc2a6a7ee31cc74205493b3b7b8051e WatchSource:0}: Error finding container 5b0e0004f563cd61bab16d26f7f72d4d2dc2a6a7ee31cc74205493b3b7b8051e: Status 404 returned error can't find the container with id 5b0e0004f563cd61bab16d26f7f72d4d2dc2a6a7ee31cc74205493b3b7b8051e
Jul 13 19:33:53.754456 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.754425    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt" event={"ID":"f5aa1169-7397-4d14-835f-1daf8ff417d5","Type":"ContainerStarted","Data":"19a11d34f8e97ffb040349bbb10c6872826158da565e18ba705ef02803423b37"}
Jul 13 19:33:53.754643 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.754475    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt" event={"ID":"f5aa1169-7397-4d14-835f-1daf8ff417d5","Type":"ContainerStarted","Data":"5b0e0004f563cd61bab16d26f7f72d4d2dc2a6a7ee31cc74205493b3b7b8051e"}
Jul 13 19:33:53.773436 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:33:53.773363    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt" podStartSLOduration=1.574386088 podStartE2EDuration="1.773345925s" totalImagesPullingTime="198.959837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:33:53.771754664 +0000 UTC m=+730.187030871" watchObservedRunningTime="2026-07-13 19:33:53.773345925 +0000 UTC m=+730.188622104"
Jul 13 19:34:00.015775 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:00.015740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:34:03.328263 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:03.328225    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:34:03.328692 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:03.328292    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:34:03.340713 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:03.340690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:34:03.808413 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:03.808392    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nbpnt"
Jul 13 19:34:58.209515 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:58.209478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:34:58.299551 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:58.299513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:34:59.415390 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:59.415354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:34:59.545168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:34:59.545123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:35:09.749398 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:35:09.749320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:36:09.905956 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:09.905922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:36:10.016902 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.016869    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-84h9n"]
Jul 13 19:36:10.017122 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.017099    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n" podUID="cd5d483d-c6dc-4ef1-9c27-4123b00600a8" containerName="manager" containerID="cri-o://be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9" gracePeriod=30
Jul 13 19:36:10.256785 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.256760    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:36:10.315491 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.315462    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-webhook-cert\") pod \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") "
Jul 13 19:36:10.315649 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.315504    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dv8m\" (UniqueName: \"kubernetes.io/projected/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-kube-api-access-6dv8m\") pod \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\" (UID: \"cd5d483d-c6dc-4ef1-9c27-4123b00600a8\") "
Jul 13 19:36:10.317641 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.317606    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-kube-api-access-6dv8m" (OuterVolumeSpecName: "kube-api-access-6dv8m") pod "cd5d483d-c6dc-4ef1-9c27-4123b00600a8" (UID: "cd5d483d-c6dc-4ef1-9c27-4123b00600a8"). InnerVolumeSpecName "kube-api-access-6dv8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:36:10.317742 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.317664    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cd5d483d-c6dc-4ef1-9c27-4123b00600a8" (UID: "cd5d483d-c6dc-4ef1-9c27-4123b00600a8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:36:10.327608 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.327583    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd5d483d-c6dc-4ef1-9c27-4123b00600a8" containerID="be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9" exitCode=0
Jul 13 19:36:10.327735 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.327635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n" event={"ID":"cd5d483d-c6dc-4ef1-9c27-4123b00600a8","Type":"ContainerDied","Data":"be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9"}
Jul 13 19:36:10.327735 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.327638    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n"
Jul 13 19:36:10.327735 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.327674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8fb89cc8-84h9n" event={"ID":"cd5d483d-c6dc-4ef1-9c27-4123b00600a8","Type":"ContainerDied","Data":"a7782b239426e9120be1294105da71f8a7f16304f5c12da1d92db8d4399f5b71"}
Jul 13 19:36:10.327735 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.327695    2562 scope.go:117] "RemoveContainer" containerID="be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9"
Jul 13 19:36:10.336712 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.336696    2562 scope.go:117] "RemoveContainer" containerID="be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9"
Jul 13 19:36:10.336974 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:36:10.336951    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9\": container with ID starting with be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9 not found: ID does not exist" containerID="be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9"
Jul 13 19:36:10.337059 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.336980    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9"} err="failed to get container status \"be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9\": rpc error: code = NotFound desc = could not find container \"be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9\": container with ID starting with be879c10b03ffd2367b661ec0100439f6481c877c2b6a4b2df8f46fb0ba30dd9 not found: ID does not exist"
Jul 13 19:36:10.348493 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.348462    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-84h9n"]
Jul 13 19:36:10.350032 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.350009    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-84h9n"]
Jul 13 19:36:10.416660 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.416637    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-webhook-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:36:10.416660 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:10.416658    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dv8m\" (UniqueName: \"kubernetes.io/projected/cd5d483d-c6dc-4ef1-9c27-4123b00600a8-kube-api-access-6dv8m\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:36:11.022300 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.022269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:36:11.156406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.153897    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-tdbfm"]
Jul 13 19:36:11.156406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.154650    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5d483d-c6dc-4ef1-9c27-4123b00600a8" containerName="manager"
Jul 13 19:36:11.156406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.154669    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5d483d-c6dc-4ef1-9c27-4123b00600a8" containerName="manager"
Jul 13 19:36:11.156406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.154843    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd5d483d-c6dc-4ef1-9c27-4123b00600a8" containerName="manager"
Jul 13 19:36:11.159862 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.159834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.162740 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.162721    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 13 19:36:11.163625 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.163609    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 13 19:36:11.163680 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.163645    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-46flg\""
Jul 13 19:36:11.166298 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.166276    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-tdbfm"]
Jul 13 19:36:11.224156 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.224125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7p4c\" (UniqueName: \"kubernetes.io/projected/f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e-kube-api-access-t7p4c\") pod \"maas-controller-7d8fb89cc8-tdbfm\" (UID: \"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e\") " pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.224339 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.224188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e-webhook-cert\") pod \"maas-controller-7d8fb89cc8-tdbfm\" (UID: \"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e\") " pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.325289 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.325249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e-webhook-cert\") pod \"maas-controller-7d8fb89cc8-tdbfm\" (UID: \"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e\") " pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.325480 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.325336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7p4c\" (UniqueName: \"kubernetes.io/projected/f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e-kube-api-access-t7p4c\") pod \"maas-controller-7d8fb89cc8-tdbfm\" (UID: \"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e\") " pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.327880 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.327856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e-webhook-cert\") pod \"maas-controller-7d8fb89cc8-tdbfm\" (UID: \"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e\") " pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.334515 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.334490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7p4c\" (UniqueName: \"kubernetes.io/projected/f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e-kube-api-access-t7p4c\") pod \"maas-controller-7d8fb89cc8-tdbfm\" (UID: \"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e\") " pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.470967 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.470946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:11.598973 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:11.598883    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8fb89cc8-tdbfm"]
Jul 13 19:36:11.601131 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:36:11.601100    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf266b68e_5fa7_48f7_a113_ea6ddb7f8f6e.slice/crio-8cdda903d0ecafb5fe7a6d271532adc2af8d95634354ddeeecf0bb82e862f9dd WatchSource:0}: Error finding container 8cdda903d0ecafb5fe7a6d271532adc2af8d95634354ddeeecf0bb82e862f9dd: Status 404 returned error can't find the container with id 8cdda903d0ecafb5fe7a6d271532adc2af8d95634354ddeeecf0bb82e862f9dd
Jul 13 19:36:12.090915 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:12.090888    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd5d483d-c6dc-4ef1-9c27-4123b00600a8" path="/var/lib/kubelet/pods/cd5d483d-c6dc-4ef1-9c27-4123b00600a8/volumes"
Jul 13 19:36:12.339600 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:12.339562    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm" event={"ID":"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e","Type":"ContainerStarted","Data":"1773d2e912e41168ef3d635c53e3eff739a4002a8ce267a84bb60618075f72f2"}
Jul 13 19:36:12.339762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:12.339629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm" event={"ID":"f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e","Type":"ContainerStarted","Data":"8cdda903d0ecafb5fe7a6d271532adc2af8d95634354ddeeecf0bb82e862f9dd"}
Jul 13 19:36:12.339762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:12.339658    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:12.398199 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:12.398117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm" podStartSLOduration=0.840009223 podStartE2EDuration="1.398096605s" totalImagesPullingTime="558.087382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:36:12.396696139 +0000 UTC m=+868.811972319" watchObservedRunningTime="2026-07-13 19:36:12.398096605 +0000 UTC m=+868.813372788"
Jul 13 19:36:23.346052 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:23.346025    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d8fb89cc8-tdbfm"
Jul 13 19:36:53.836324 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:53.836287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:36:53.932396 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:53.932363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:36:54.062049 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:36:54.062015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:37:00.186832 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:37:00.186794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:37:03.277648 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:37:03.277613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:37:06.358250 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:37:06.358210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:37:06.431799 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:37:06.431759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:38:04.916012 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:38:04.915934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:38:21.296843 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:38:21.296813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:39:34.036721 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:39:34.036686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:40:02.904691 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:40:02.904613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:41:00.872935 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:41:00.872901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:41:00.978242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:41:00.978209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:43:34.803582 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:43:34.803526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:45:58.020149 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:45:58.020077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:12.691941 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:12.691898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:17.847946 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:17.847907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:18.168442 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:18.168348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:26.089707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:26.089642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:31.211388 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:31.211357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:34.216965 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:34.216932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:34.317798 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:34.317757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:34.673662 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:34.673585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:40.599222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:40.599186    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:42.012455 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.012420    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"]
Jul 13 19:48:42.015813 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.015796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.019222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.019199    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\""
Jul 13 19:48:42.019303 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.019199    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1b0d941e-serving-cert\""
Jul 13 19:48:42.029664 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.029641    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"]
Jul 13 19:48:42.075577 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.075516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.075746 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.075718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8115f40-51f9-4c06-b094-84eea01c26a8-openshift-service-ca\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.075809 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.075769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2pch\" (UniqueName: \"kubernetes.io/projected/c8115f40-51f9-4c06-b094-84eea01c26a8-kube-api-access-d2pch\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.176439 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.176405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2pch\" (UniqueName: \"kubernetes.io/projected/c8115f40-51f9-4c06-b094-84eea01c26a8-kube-api-access-d2pch\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.176638 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.176470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.176638 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.176511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8115f40-51f9-4c06-b094-84eea01c26a8-openshift-service-ca\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.176638 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:48:42.176631    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-serving-cert: secret "maas-api-e2e-ait-1b0d941e-serving-cert" not found
Jul 13 19:48:42.176765 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:48:42.176711    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls podName:c8115f40-51f9-4c06-b094-84eea01c26a8 nodeName:}" failed. No retries permitted until 2026-07-13 19:48:42.67669205 +0000 UTC m=+1619.091968210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls") pod "maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" (UID: "c8115f40-51f9-4c06-b094-84eea01c26a8") : secret "maas-api-e2e-ait-1b0d941e-serving-cert" not found
Jul 13 19:48:42.177175 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.177157    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8115f40-51f9-4c06-b094-84eea01c26a8-openshift-service-ca\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.189387 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.189358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2pch\" (UniqueName: \"kubernetes.io/projected/c8115f40-51f9-4c06-b094-84eea01c26a8-kube-api-access-d2pch\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.681791 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.681758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.684342 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.684304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls\") pod \"maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:42.925811 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:42.925773    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:43.056370 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:43.056346    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"]
Jul 13 19:48:43.058757 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:48:43.058730    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8115f40_51f9_4c06_b094_84eea01c26a8.slice/crio-6b183ba56ae57cd9860b54a566b98b42d79f1fd97c8034bd6ab75fb0c1c97fab WatchSource:0}: Error finding container 6b183ba56ae57cd9860b54a566b98b42d79f1fd97c8034bd6ab75fb0c1c97fab: Status 404 returned error can't find the container with id 6b183ba56ae57cd9860b54a566b98b42d79f1fd97c8034bd6ab75fb0c1c97fab
Jul 13 19:48:43.060038 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:43.060020    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:48:43.471959 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:43.471928    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:43.521763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:43.521722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" event={"ID":"c8115f40-51f9-4c06-b094-84eea01c26a8","Type":"ContainerStarted","Data":"6b183ba56ae57cd9860b54a566b98b42d79f1fd97c8034bd6ab75fb0c1c97fab"}
Jul 13 19:48:43.571801 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:43.571768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:44.527182 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:44.527149    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" event={"ID":"c8115f40-51f9-4c06-b094-84eea01c26a8","Type":"ContainerStarted","Data":"2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493"}
Jul 13 19:48:44.527611 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:44.527210    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:48:44.544853 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:44.544804    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" podStartSLOduration=2.959155993 podStartE2EDuration="3.54478638s" totalImagesPullingTime="585.630387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:48:44.543841007 +0000 UTC m=+1620.959117188" watchObservedRunningTime="2026-07-13 19:48:44.54478638 +0000 UTC m=+1620.960062562"
Jul 13 19:48:44.656310 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:44.656260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:48:50.538200 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:48:50.538165    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:49:22.873208 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:22.873171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:49:22.933998 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:22.933974    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:49:57.449687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.449652    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"]
Jul 13 19:49:57.450285 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.450132    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" podUID="c8115f40-51f9-4c06-b094-84eea01c26a8" containerName="maas-api" containerID="cri-o://2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493" gracePeriod=30
Jul 13 19:49:57.477678 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.477653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:49:57.696979 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.696958    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:49:57.782884 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.782813    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls\") pod \"c8115f40-51f9-4c06-b094-84eea01c26a8\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") "
Jul 13 19:49:57.782884 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.782848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8115f40-51f9-4c06-b094-84eea01c26a8-openshift-service-ca\") pod \"c8115f40-51f9-4c06-b094-84eea01c26a8\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") "
Jul 13 19:49:57.783069 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.782897    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2pch\" (UniqueName: \"kubernetes.io/projected/c8115f40-51f9-4c06-b094-84eea01c26a8-kube-api-access-d2pch\") pod \"c8115f40-51f9-4c06-b094-84eea01c26a8\" (UID: \"c8115f40-51f9-4c06-b094-84eea01c26a8\") "
Jul 13 19:49:57.783218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.783195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8115f40-51f9-4c06-b094-84eea01c26a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8115f40-51f9-4c06-b094-84eea01c26a8" (UID: "c8115f40-51f9-4c06-b094-84eea01c26a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:49:57.785051 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.785016    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8115f40-51f9-4c06-b094-84eea01c26a8" (UID: "c8115f40-51f9-4c06-b094-84eea01c26a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:49:57.785135 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.785070    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8115f40-51f9-4c06-b094-84eea01c26a8-kube-api-access-d2pch" (OuterVolumeSpecName: "kube-api-access-d2pch") pod "c8115f40-51f9-4c06-b094-84eea01c26a8" (UID: "c8115f40-51f9-4c06-b094-84eea01c26a8"). InnerVolumeSpecName "kube-api-access-d2pch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:49:57.845076 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.845039    2562 generic.go:356] "Generic (PLEG): container finished" podID="c8115f40-51f9-4c06-b094-84eea01c26a8" containerID="2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493" exitCode=0
Jul 13 19:49:57.845180 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.845082    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"
Jul 13 19:49:57.845180 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.845125    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" event={"ID":"c8115f40-51f9-4c06-b094-84eea01c26a8","Type":"ContainerDied","Data":"2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493"}
Jul 13 19:49:57.845180 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.845171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt" event={"ID":"c8115f40-51f9-4c06-b094-84eea01c26a8","Type":"ContainerDied","Data":"6b183ba56ae57cd9860b54a566b98b42d79f1fd97c8034bd6ab75fb0c1c97fab"}
Jul 13 19:49:57.845306 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.845194    2562 scope.go:117] "RemoveContainer" containerID="2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493"
Jul 13 19:49:57.853984 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.853968    2562 scope.go:117] "RemoveContainer" containerID="2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493"
Jul 13 19:49:57.854228 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:49:57.854211    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493\": container with ID starting with 2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493 not found: ID does not exist" containerID="2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493"
Jul 13 19:49:57.854276 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.854243    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493"} err="failed to get container status \"2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493\": rpc error: code = NotFound desc = could not find container \"2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493\": container with ID starting with 2f485e5ceb46043323d0b7ce00c2ea6fd3ffd5119c8c451a754a90eea46f4493 not found: ID does not exist"
Jul 13 19:49:57.864740 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.864720    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"]
Jul 13 19:49:57.870159 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.870140    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b0d941e-655ccd4cbb-f2ztt"]
Jul 13 19:49:57.883828 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.883801    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8115f40-51f9-4c06-b094-84eea01c26a8-maas-api-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:49:57.883828 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.883826    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8115f40-51f9-4c06-b094-84eea01c26a8-openshift-service-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:49:57.883944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:57.883841    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2pch\" (UniqueName: \"kubernetes.io/projected/c8115f40-51f9-4c06-b094-84eea01c26a8-kube-api-access-d2pch\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:49:58.090586 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:49:58.090524    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8115f40-51f9-4c06-b094-84eea01c26a8" path="/var/lib/kubelet/pods/c8115f40-51f9-4c06-b094-84eea01c26a8/volumes"
Jul 13 19:50:02.126768 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:02.126733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:07.834295 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:07.834218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:07.934957 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:07.934927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:12.550983 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.550944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:12.741783 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.741744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:12.837704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.837669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:12.882591 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.882515    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"]
Jul 13 19:50:12.882960 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.882942    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8115f40-51f9-4c06-b094-84eea01c26a8" containerName="maas-api"
Jul 13 19:50:12.882960 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.882962    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8115f40-51f9-4c06-b094-84eea01c26a8" containerName="maas-api"
Jul 13 19:50:12.883136 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.883038    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8115f40-51f9-4c06-b094-84eea01c26a8" containerName="maas-api"
Jul 13 19:50:12.887874 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.887854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:12.890489 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.890469    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f3b1e94a-gw-openshift-default-dockercfg-4ps6k\""
Jul 13 19:50:12.898806 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.898783    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"]
Jul 13 19:50:12.929491 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:12.929466    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:13.011404 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-credential-socket\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.011686 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-data\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.011817 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa934414-c727-4b0e-8703-7217bd071fd1-istiod-ca-cert\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.011817 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-envoy\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.011817 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011785    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-socket\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.011997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011834    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkcg7\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-kube-api-access-mkcg7\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.011997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.011977    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-istio-token\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.012101 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.012076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa934414-c727-4b0e-8703-7217bd071fd1-istio-podinfo\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.012167 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.012133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-certs\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.112689 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-data\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.112689 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa934414-c727-4b0e-8703-7217bd071fd1-istiod-ca-cert\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.112689 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112684    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-envoy\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.112906 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-socket\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.112906 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcg7\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-kube-api-access-mkcg7\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.112906 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-istio-token\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa934414-c727-4b0e-8703-7217bd071fd1-istio-podinfo\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-certs\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.112999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-credential-socket\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.113029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-data\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113229 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.113102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-socket\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113310 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.113288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-credential-socket\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113367 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.113311    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-certs\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.113438 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.113421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa934414-c727-4b0e-8703-7217bd071fd1-istiod-ca-cert\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.115164 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.115141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-envoy\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.115449 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.115431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa934414-c727-4b0e-8703-7217bd071fd1-istio-podinfo\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.121610 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.121586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkcg7\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-kube-api-access-mkcg7\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.121918 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.121897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-istio-token\") pod \"e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") " pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.199466 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.199444    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:13.333227 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.333198    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"]
Jul 13 19:50:13.335470 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:50:13.335434    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa934414_c727_4b0e_8703_7217bd071fd1.slice/crio-ad0cff6b72dd3a6825b01efed0e1c2a75266c830ef278404430d80c813d6ac82 WatchSource:0}: Error finding container ad0cff6b72dd3a6825b01efed0e1c2a75266c830ef278404430d80c813d6ac82: Status 404 returned error can't find the container with id ad0cff6b72dd3a6825b01efed0e1c2a75266c830ef278404430d80c813d6ac82
Jul 13 19:50:13.920837 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:13.920796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" event={"ID":"fa934414-c727-4b0e-8703-7217bd071fd1","Type":"ContainerStarted","Data":"ad0cff6b72dd3a6825b01efed0e1c2a75266c830ef278404430d80c813d6ac82"}
Jul 13 19:50:14.192874 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.192792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:14.262821 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.262788    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"]
Jul 13 19:50:14.266668 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.266639    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.269023 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.269003    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-f3b1e94a-serving-cert\""
Jul 13 19:50:14.269137 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.269042    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\""
Jul 13 19:50:14.277166 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.277146    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"]
Jul 13 19:50:14.304705 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.304682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:14.322263 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.322238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssnhq\" (UniqueName: \"kubernetes.io/projected/bb232493-f8d4-4efb-9afe-a945f445b671-kube-api-access-ssnhq\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.322350 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.322284    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb232493-f8d4-4efb-9afe-a945f445b671-openshift-service-ca\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.322389 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.322372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.423169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.423140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.423351 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.423203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssnhq\" (UniqueName: \"kubernetes.io/projected/bb232493-f8d4-4efb-9afe-a945f445b671-kube-api-access-ssnhq\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.423351 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.423236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb232493-f8d4-4efb-9afe-a945f445b671-openshift-service-ca\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.423351 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:50:14.423309    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-serving-cert: secret "maas-api-e2e-migrate-f3b1e94a-serving-cert" not found
Jul 13 19:50:14.423515 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:50:14.423398    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls podName:bb232493-f8d4-4efb-9afe-a945f445b671 nodeName:}" failed. No retries permitted until 2026-07-13 19:50:14.923376143 +0000 UTC m=+1711.338652304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls") pod "maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" (UID: "bb232493-f8d4-4efb-9afe-a945f445b671") : secret "maas-api-e2e-migrate-f3b1e94a-serving-cert" not found
Jul 13 19:50:14.423865 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.423846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb232493-f8d4-4efb-9afe-a945f445b671-openshift-service-ca\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.432072 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.432049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssnhq\" (UniqueName: \"kubernetes.io/projected/bb232493-f8d4-4efb-9afe-a945f445b671-kube-api-access-ssnhq\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.884014 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.883982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:14.926581 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.926530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.928935 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.928915    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls\") pod \"maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:14.985738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:14.985716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:15.179497 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:15.179416    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:15.331376 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:15.331348    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"]
Jul 13 19:50:15.334750 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:50:15.334718    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb232493_f8d4_4efb_9afe_a945f445b671.slice/crio-b17daa9799adfe06fd0bb81fc079356d0ae22b062aa2047a036027ecd80be789 WatchSource:0}: Error finding container b17daa9799adfe06fd0bb81fc079356d0ae22b062aa2047a036027ecd80be789: Status 404 returned error can't find the container with id b17daa9799adfe06fd0bb81fc079356d0ae22b062aa2047a036027ecd80be789
Jul 13 19:50:15.933658 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:15.933616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" event={"ID":"bb232493-f8d4-4efb-9afe-a945f445b671","Type":"ContainerStarted","Data":"b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9"}
Jul 13 19:50:15.934100 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:15.933677    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" event={"ID":"bb232493-f8d4-4efb-9afe-a945f445b671","Type":"ContainerStarted","Data":"b17daa9799adfe06fd0bb81fc079356d0ae22b062aa2047a036027ecd80be789"}
Jul 13 19:50:15.934100 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:15.933770    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:15.950939 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:15.950840    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" podStartSLOduration=1.491851371 podStartE2EDuration="1.95081886s" totalImagesPullingTime="458.967489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:50:15.949396814 +0000 UTC m=+1712.364672997" watchObservedRunningTime="2026-07-13 19:50:15.95081886 +0000 UTC m=+1712.366095042"
Jul 13 19:50:17.072380 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:17.072344    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:50:17.072771 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:17.072424    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:50:17.072771 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:17.072467    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:50:17.944203 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:17.944118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" event={"ID":"fa934414-c727-4b0e-8703-7217bd071fd1","Type":"ContainerStarted","Data":"2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"}
Jul 13 19:50:17.964465 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:17.964417    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podStartSLOduration=2.229788554 podStartE2EDuration="5.964400834s" totalImagesPullingTime="3.73461228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:50:17.963017011 +0000 UTC m=+1714.378293197" watchObservedRunningTime="2026-07-13 19:50:17.964400834 +0000 UTC m=+1714.379677082"
Jul 13 19:50:18.200212 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:18.200139    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:18.204666 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:18.204644    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:18.948203 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:18.948169    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:18.949199 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:18.949179    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:21.943863 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:21.943835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:50:24.174725 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:24.174673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:24.239011 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:24.238961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:27.424173 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:27.424132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:50:30.001263 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:30.001200    2562 generic.go:356] "Generic (PLEG): container finished" podID="fa934414-c727-4b0e-8703-7217bd071fd1" containerID="2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea" exitCode=0
Jul 13 19:50:30.001263 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:30.001254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" event={"ID":"fa934414-c727-4b0e-8703-7217bd071fd1","Type":"ContainerDied","Data":"2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"}
Jul 13 19:50:30.001680 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:30.001614    2562 scope.go:117] "RemoveContainer" containerID="2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"
Jul 13 19:50:30.003591 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:30.003553    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:50:30.003695 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:30.003650    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:50:30.003761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:30.003702    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:50:31.008026 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:31.007993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" event={"ID":"fa934414-c727-4b0e-8703-7217bd071fd1","Type":"ContainerStarted","Data":"0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e"}
Jul 13 19:50:31.200482 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:31.200443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:31.201501 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:31.201466    2562 patch_prober.go:28] interesting pod/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 13 19:50:31.201584 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:31.201556    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 13 19:50:32.200476 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:32.200438    2562 patch_prober.go:28] interesting pod/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 13 19:50:32.200867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:32.200504    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 13 19:50:33.200406 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:33.200370    2562 patch_prober.go:28] interesting pod/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 13 19:50:33.200624 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:33.200433    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 13 19:50:33.949102 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:33.949069    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:34.200938 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:34.200845    2562 patch_prober.go:28] interesting pod/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 13 19:50:34.200938 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:34.200916    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 13 19:50:35.204615 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:35.204589    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:50:35.205677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:50:35.205659    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:51:18.820011 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:18.819977    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"]
Jul 13 19:51:18.820567 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:18.820185    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy" containerID="cri-o://0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e" gracePeriod=30
Jul 13 19:51:18.845043 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:18.845019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:18.974723 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:18.974683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:20.207027 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:20.206986    2562 patch_prober.go:28] interesting pod/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 13 19:51:20.207382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:20.207056    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:51:24.079054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.079001    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:51:24.091367 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-credential-socket\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091379    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkcg7\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-kube-api-access-mkcg7\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091397    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-istio-token\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091421    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-socket\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091477    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa934414-c727-4b0e-8703-7217bd071fd1-istiod-ca-cert\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091512    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-envoy\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091556    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-data\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091580    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa934414-c727-4b0e-8703-7217bd071fd1-istio-podinfo\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091610    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-certs\") pod \"fa934414-c727-4b0e-8703-7217bd071fd1\" (UID: \"fa934414-c727-4b0e-8703-7217bd071fd1\") "
Jul 13 19:51:24.091714 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091687    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:51:24.091948 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091869    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.091948 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091901    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:51:24.091948 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.091913    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa934414-c727-4b0e-8703-7217bd071fd1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:51:24.092396 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.092272    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:51:24.092396 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.092382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:51:24.094150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.094126    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa934414-c727-4b0e-8703-7217bd071fd1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:51:24.094150 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.094142    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-kube-api-access-mkcg7" (OuterVolumeSpecName: "kube-api-access-mkcg7") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "kube-api-access-mkcg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:51:24.094283 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.094260    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:51:24.094365 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.094345    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa934414-c727-4b0e-8703-7217bd071fd1" (UID: "fa934414-c727-4b0e-8703-7217bd071fd1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:51:24.192468 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192441    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192468 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192467    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192475    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa934414-c727-4b0e-8703-7217bd071fd1-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192483    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192492    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkcg7\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-kube-api-access-mkcg7\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192500    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa934414-c727-4b0e-8703-7217bd071fd1-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192508    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa934414-c727-4b0e-8703-7217bd071fd1-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.192640 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.192515    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa934414-c727-4b0e-8703-7217bd071fd1-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.235430 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.235401    2562 generic.go:356] "Generic (PLEG): container finished" podID="fa934414-c727-4b0e-8703-7217bd071fd1" containerID="0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e" exitCode=0
Jul 13 19:51:24.235586 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.235451    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"
Jul 13 19:51:24.235586 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.235478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" event={"ID":"fa934414-c727-4b0e-8703-7217bd071fd1","Type":"ContainerDied","Data":"0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e"}
Jul 13 19:51:24.235586 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.235529    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z" event={"ID":"fa934414-c727-4b0e-8703-7217bd071fd1","Type":"ContainerDied","Data":"ad0cff6b72dd3a6825b01efed0e1c2a75266c830ef278404430d80c813d6ac82"}
Jul 13 19:51:24.235586 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.235573    2562 scope.go:117] "RemoveContainer" containerID="0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e"
Jul 13 19:51:24.248972 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.248946    2562 scope.go:117] "RemoveContainer" containerID="2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"
Jul 13 19:51:24.256820 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.256799    2562 scope.go:117] "RemoveContainer" containerID="0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e"
Jul 13 19:51:24.257161 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:51:24.257131    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e\": container with ID starting with 0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e not found: ID does not exist" containerID="0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e"
Jul 13 19:51:24.257212 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.257180    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e"} err="failed to get container status \"0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e\": rpc error: code = NotFound desc = could not find container \"0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e\": container with ID starting with 0142cd0ea9100ebc813e9a5fb88e5f9093ce8799ac08f51b61bb2afc5147b49e not found: ID does not exist"
Jul 13 19:51:24.257255 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.257221    2562 scope.go:117] "RemoveContainer" containerID="2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"
Jul 13 19:51:24.257561 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:51:24.257519    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea\": container with ID starting with 2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea not found: ID does not exist" containerID="2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"
Jul 13 19:51:24.257653 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.257593    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea"} err="failed to get container status \"2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea\": rpc error: code = NotFound desc = could not find container \"2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea\": container with ID starting with 2273b415ecb8e78082d8b740c719b620237b8cbef500e3d8be4ae6ffb8fe61ea not found: ID does not exist"
Jul 13 19:51:24.259965 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.259942    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"]
Jul 13 19:51:24.266002 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.265984    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f3b1e94a-gw-openshift-default-5658f79b95-lrb4z"]
Jul 13 19:51:24.535204 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.535120    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"]
Jul 13 19:51:24.535393 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.535364    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" podUID="bb232493-f8d4-4efb-9afe-a945f445b671" containerName="maas-api" containerID="cri-o://b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9" gracePeriod=30
Jul 13 19:51:24.562707 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.562683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:24.777771 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.777741    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:51:24.796648 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.796588    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls\") pod \"bb232493-f8d4-4efb-9afe-a945f445b671\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") "
Jul 13 19:51:24.796648 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.796641    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb232493-f8d4-4efb-9afe-a945f445b671-openshift-service-ca\") pod \"bb232493-f8d4-4efb-9afe-a945f445b671\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") "
Jul 13 19:51:24.797193 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.796889    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssnhq\" (UniqueName: \"kubernetes.io/projected/bb232493-f8d4-4efb-9afe-a945f445b671-kube-api-access-ssnhq\") pod \"bb232493-f8d4-4efb-9afe-a945f445b671\" (UID: \"bb232493-f8d4-4efb-9afe-a945f445b671\") "
Jul 13 19:51:24.797193 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.796953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb232493-f8d4-4efb-9afe-a945f445b671-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb232493-f8d4-4efb-9afe-a945f445b671" (UID: "bb232493-f8d4-4efb-9afe-a945f445b671"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:51:24.797193 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.797159    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb232493-f8d4-4efb-9afe-a945f445b671-openshift-service-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.799109 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.799084    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb232493-f8d4-4efb-9afe-a945f445b671" (UID: "bb232493-f8d4-4efb-9afe-a945f445b671"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:51:24.800037 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.800015    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb232493-f8d4-4efb-9afe-a945f445b671-kube-api-access-ssnhq" (OuterVolumeSpecName: "kube-api-access-ssnhq") pod "bb232493-f8d4-4efb-9afe-a945f445b671" (UID: "bb232493-f8d4-4efb-9afe-a945f445b671"). InnerVolumeSpecName "kube-api-access-ssnhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:51:24.897482 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.897459    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssnhq\" (UniqueName: \"kubernetes.io/projected/bb232493-f8d4-4efb-9afe-a945f445b671-kube-api-access-ssnhq\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.897482 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.897483    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb232493-f8d4-4efb-9afe-a945f445b671-maas-api-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:51:24.975744 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:24.975717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:25.243237 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.243191    2562 generic.go:356] "Generic (PLEG): container finished" podID="bb232493-f8d4-4efb-9afe-a945f445b671" containerID="b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9" exitCode=0
Jul 13 19:51:25.243237 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.243239    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"
Jul 13 19:51:25.243761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.243254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" event={"ID":"bb232493-f8d4-4efb-9afe-a945f445b671","Type":"ContainerDied","Data":"b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9"}
Jul 13 19:51:25.243761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.243316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x" event={"ID":"bb232493-f8d4-4efb-9afe-a945f445b671","Type":"ContainerDied","Data":"b17daa9799adfe06fd0bb81fc079356d0ae22b062aa2047a036027ecd80be789"}
Jul 13 19:51:25.243761 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.243349    2562 scope.go:117] "RemoveContainer" containerID="b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9"
Jul 13 19:51:25.253681 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.253665    2562 scope.go:117] "RemoveContainer" containerID="b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9"
Jul 13 19:51:25.253966 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:51:25.253945    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9\": container with ID starting with b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9 not found: ID does not exist" containerID="b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9"
Jul 13 19:51:25.254021 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.253973    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9"} err="failed to get container status \"b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9\": rpc error: code = NotFound desc = could not find container \"b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9\": container with ID starting with b33ea48414d16cf75cb5fcebb84b1ac251f005861f9a6031310a2f73b4ac0aa9 not found: ID does not exist"
Jul 13 19:51:25.266407 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.266383    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"]
Jul 13 19:51:25.271901 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.271882    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f3b1e94a-7887ff985d-6t46x"]
Jul 13 19:51:25.273986 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:25.273968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:26.090776 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:26.090746    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb232493-f8d4-4efb-9afe-a945f445b671" path="/var/lib/kubelet/pods/bb232493-f8d4-4efb-9afe-a945f445b671/volumes"
Jul 13 19:51:26.091146 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:26.091133    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" path="/var/lib/kubelet/pods/fa934414-c727-4b0e-8703-7217bd071fd1/volumes"
Jul 13 19:51:30.687689 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.687654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:30.758219 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.758183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:30.907272 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907248    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"]
Jul 13 19:51:30.907691 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907670    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb232493-f8d4-4efb-9afe-a945f445b671" containerName="maas-api"
Jul 13 19:51:30.907691 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907690    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb232493-f8d4-4efb-9afe-a945f445b671" containerName="maas-api"
Jul 13 19:51:30.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907705    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy"
Jul 13 19:51:30.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907710    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy"
Jul 13 19:51:30.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907721    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy"
Jul 13 19:51:30.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907729    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy"
Jul 13 19:51:30.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907804    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy"
Jul 13 19:51:30.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.907816    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb232493-f8d4-4efb-9afe-a945f445b671" containerName="maas-api"
Jul 13 19:51:30.912109 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.912094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.914890 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.914870    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-32278165-openshift-default-dockercfg-8k768\""
Jul 13 19:51:30.928173 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.928153    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"]
Jul 13 19:51:30.936467 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-podinfo\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936582 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-token\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936582 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-certs\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936583    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-data\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6m27\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-kube-api-access-t6m27\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936650    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-socket\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-credential-socket\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936814 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-envoy\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.936814 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.936775    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istiod-ca-cert\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:30.949958 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:30.949904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:31.037671 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istiod-ca-cert\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.037879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037697    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-podinfo\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.037879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-token\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.037879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-certs\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.037879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-data\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.037879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6m27\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-kube-api-access-t6m27\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.037879 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-socket\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.038183 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-credential-socket\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.038183 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.037930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-envoy\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.038664 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.038619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-certs\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.038964 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.038934    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istiod-ca-cert\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.039054 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.039003    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-credential-socket\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.039094 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.039045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-socket\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.039094 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.039072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-data\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.040650 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.040630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-envoy\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.040745 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.040726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-podinfo\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.047838 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.047816    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-token\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.053449 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.053418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:31.054033 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.054016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6m27\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-kube-api-access-t6m27\") pod \"e2e-ait-32278165-openshift-default-7796c799bc-6qhzw\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") " pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.222155 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.222069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:31.362082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.362058    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"]
Jul 13 19:51:31.364146 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:51:31.364109    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9421b4b8_fe73_489e_8a5b_81c01cae23ab.slice/crio-509041364b7cab3748321c84650913692432e33690e2e783d675d0314b1bcea4 WatchSource:0}: Error finding container 509041364b7cab3748321c84650913692432e33690e2e783d675d0314b1bcea4: Status 404 returned error can't find the container with id 509041364b7cab3748321c84650913692432e33690e2e783d675d0314b1bcea4
Jul 13 19:51:31.366147 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.366116    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:51:31.366242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.366175    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:51:31.366242 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:31.366212    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:51:32.135222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.135156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:32.238309 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.238277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:32.278574 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.278529    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" event={"ID":"9421b4b8-fe73-489e-8a5b-81c01cae23ab","Type":"ContainerStarted","Data":"31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"}
Jul 13 19:51:32.278742 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.278600    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" event={"ID":"9421b4b8-fe73-489e-8a5b-81c01cae23ab","Type":"ContainerStarted","Data":"509041364b7cab3748321c84650913692432e33690e2e783d675d0314b1bcea4"}
Jul 13 19:51:32.312432 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.312389    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" podStartSLOduration=2.312370111 podStartE2EDuration="2.312370111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:51:32.310081448 +0000 UTC m=+1788.725357630" watchObservedRunningTime="2026-07-13 19:51:32.312370111 +0000 UTC m=+1788.727646294"
Jul 13 19:51:32.340695 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.340668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:32.469578 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:32.469483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:33.196849 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:33.196814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:33.222736 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:33.222710    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:33.227752 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:33.227731    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:33.283784 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:33.283755    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:33.284588 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:33.284567    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:33.297088 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:33.297065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:34.225375 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:34.225347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:35.236959 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:35.236869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:35.338273 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:35.338237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:37.773120 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:37.773083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:37.871529 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:37.871491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:51:43.332182 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:43.332146    2562 generic.go:356] "Generic (PLEG): container finished" podID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerID="31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2" exitCode=0
Jul 13 19:51:43.332702 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:43.332227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" event={"ID":"9421b4b8-fe73-489e-8a5b-81c01cae23ab","Type":"ContainerDied","Data":"31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"}
Jul 13 19:51:43.332768 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:43.332727    2562 scope.go:117] "RemoveContainer" containerID="31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"
Jul 13 19:51:43.333908 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:43.333880    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:51:43.334002 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:43.333948    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:51:43.334002 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:43.333988    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:51:44.342698 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:44.341783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" event={"ID":"9421b4b8-fe73-489e-8a5b-81c01cae23ab","Type":"ContainerStarted","Data":"e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313"}
Jul 13 19:51:45.222600 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:45.222526    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:45.223443 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:45.223418    2562 patch_prober.go:28] interesting pod/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 13 19:51:45.223565 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:45.223468    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 13 19:51:46.222738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:46.222700    2562 patch_prober.go:28] interesting pod/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 13 19:51:46.223126 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:46.222757    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 13 19:51:47.223258 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:47.223223    2562 patch_prober.go:28] interesting pod/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 13 19:51:47.223658 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:47.223277    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 13 19:51:48.226475 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:48.226445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:48.226938 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:48.226754    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:51:48.227488 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:51:48.227475    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:52:47.678736 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:52:47.678698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:52:52.314367 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:52:52.314335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:52:58.019093 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:52:58.019057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:52:58.119675 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:52:58.119641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:52:59.589097 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:52:59.589057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:03.337751 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.337714    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"]
Jul 13 19:53:03.338161 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.337913    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy" containerID="cri-o://e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313" gracePeriod=30
Jul 13 19:53:03.367446 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.367415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:03.644885 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.644799    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"]
Jul 13 19:53:03.645344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.645323    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa934414-c727-4b0e-8703-7217bd071fd1" containerName="istio-proxy"
Jul 13 19:53:03.647131 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.647114    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.649713 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.649692    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7a28704c-openshift-default-dockercfg-l9r2r\""
Jul 13 19:53:03.659566 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.659522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"]
Jul 13 19:53:03.680057 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.680033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:03.732710 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-socket\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.732866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732733    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-token\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.732866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-podinfo\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.732866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732816    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-envoy\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.732866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istiod-ca-cert\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.732866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-credential-socket\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.733049 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-certs\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.733049 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd2k8\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-kube-api-access-xd2k8\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.733049 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.732950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-data\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834129 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-envoy\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834129 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istiod-ca-cert\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-credential-socket\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-certs\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd2k8\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-kube-api-access-xd2k8\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834569 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-data\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834569 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834491    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-credential-socket\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834569 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-socket\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-token\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-podinfo\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.834734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834600    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-certs\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.835085 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834786    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-socket\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.835085 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-data\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.835085 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.834945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istiod-ca-cert\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.836628 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.836608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-envoy\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.837029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.837014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-podinfo\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.844158 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.844136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd2k8\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-kube-api-access-xd2k8\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.844252 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.844195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-token\") pod \"e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") " pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:03.959850 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:03.959751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:04.101227 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.101205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"]
Jul 13 19:53:04.103325 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:53:04.103291    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d43a86f_7615_4134_a476_a6dcd11ba3e3.slice/crio-f470a3010d9496ec14a573b329189a3f8c59e83af55e7566861f237718928d6b WatchSource:0}: Error finding container f470a3010d9496ec14a573b329189a3f8c59e83af55e7566861f237718928d6b: Status 404 returned error can't find the container with id f470a3010d9496ec14a573b329189a3f8c59e83af55e7566861f237718928d6b
Jul 13 19:53:04.105413 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.105376    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:53:04.105484 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.105463    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:53:04.105527 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.105505    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:53:04.709218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.709125    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" event={"ID":"9d43a86f-7615-4134-a476-a6dcd11ba3e3","Type":"ContainerStarted","Data":"36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"}
Jul 13 19:53:04.709218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.709186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" event={"ID":"9d43a86f-7615-4134-a476-a6dcd11ba3e3","Type":"ContainerStarted","Data":"f470a3010d9496ec14a573b329189a3f8c59e83af55e7566861f237718928d6b"}
Jul 13 19:53:04.734335 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.734282    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podStartSLOduration=1.7342670610000002 podStartE2EDuration="1.734267061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:53:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:53:04.733627567 +0000 UTC m=+1881.148903752" watchObservedRunningTime="2026-07-13 19:53:04.734267061 +0000 UTC m=+1881.149543244"
Jul 13 19:53:04.786606 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.786568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:04.886282 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.886241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:04.960338 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:04.960241    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:05.038202 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:05.038168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:05.212258 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:05.212173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:05.644720 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:05.644679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:05.746444 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:05.746413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:05.965381 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:05.965301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:06.719136 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:06.719103    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:06.719919 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:06.719897    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:08.586865 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.586836    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:53:08.677814 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677781    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-envoy\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.677980 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677820    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istiod-ca-cert\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.677980 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-podinfo\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.677980 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677879    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-certs\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.677980 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677924    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-token\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.677980 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677973    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-data\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.678238 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.677994    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-credential-socket\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.678291 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678218    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:53:08.678348 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678299    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:53:08.678348 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678304    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:53:08.678451 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6m27\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-kube-api-access-t6m27\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.678508 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-socket\") pod \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\" (UID: \"9421b4b8-fe73-489e-8a5b-81c01cae23ab\") "
Jul 13 19:53:08.678811 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678773    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:53:08.678925 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:53:08.678925 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678844    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.678925 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678876    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.678925 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.678891    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.680646 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.680615    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:53:08.680760 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.680652    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:53:08.680760 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.680653    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:53:08.681194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.681170    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-kube-api-access-t6m27" (OuterVolumeSpecName: "kube-api-access-t6m27") pod "9421b4b8-fe73-489e-8a5b-81c01cae23ab" (UID: "9421b4b8-fe73-489e-8a5b-81c01cae23ab"). InnerVolumeSpecName "kube-api-access-t6m27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:53:08.729614 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.729588    2562 generic.go:356] "Generic (PLEG): container finished" podID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerID="e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313" exitCode=0
Jul 13 19:53:08.729736 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.729639    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"
Jul 13 19:53:08.729736 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.729682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" event={"ID":"9421b4b8-fe73-489e-8a5b-81c01cae23ab","Type":"ContainerDied","Data":"e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313"}
Jul 13 19:53:08.729819 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.729738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw" event={"ID":"9421b4b8-fe73-489e-8a5b-81c01cae23ab","Type":"ContainerDied","Data":"509041364b7cab3748321c84650913692432e33690e2e783d675d0314b1bcea4"}
Jul 13 19:53:08.729819 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.729760    2562 scope.go:117] "RemoveContainer" containerID="e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313"
Jul 13 19:53:08.739098 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.739083    2562 scope.go:117] "RemoveContainer" containerID="31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"
Jul 13 19:53:08.746728 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.746714    2562 scope.go:117] "RemoveContainer" containerID="e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313"
Jul 13 19:53:08.746967 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:53:08.746951    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313\": container with ID starting with e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313 not found: ID does not exist" containerID="e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313"
Jul 13 19:53:08.747028 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.746973    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313"} err="failed to get container status \"e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313\": rpc error: code = NotFound desc = could not find container \"e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313\": container with ID starting with e219862b79598f0a1d8b82d541257b3099581ce026fdaa64bd6ed3a1c6a60313 not found: ID does not exist"
Jul 13 19:53:08.747028 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.746987    2562 scope.go:117] "RemoveContainer" containerID="31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"
Jul 13 19:53:08.747198 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:53:08.747180    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2\": container with ID starting with 31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2 not found: ID does not exist" containerID="31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"
Jul 13 19:53:08.747236 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.747205    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2"} err="failed to get container status \"31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2\": rpc error: code = NotFound desc = could not find container \"31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2\": container with ID starting with 31e378f2c48c92b094fb38facfc85719ad9cca81ee3044d7ff055c2024311df2 not found: ID does not exist"
Jul 13 19:53:08.752629 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.752608    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"]
Jul 13 19:53:08.757867 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.757848    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-32278165-openshift-default-7796c799bc-6qhzw"]
Jul 13 19:53:08.780299 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.780279    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.780299 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.780297    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.780408 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.780306    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.780408 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.780314    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.780408 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.780321    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421b4b8-fe73-489e-8a5b-81c01cae23ab-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:08.780408 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:08.780330    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6m27\" (UniqueName: \"kubernetes.io/projected/9421b4b8-fe73-489e-8a5b-81c01cae23ab-kube-api-access-t6m27\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:53:10.090875 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:10.090841    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" path="/var/lib/kubelet/pods/9421b4b8-fe73-489e-8a5b-81c01cae23ab/volumes"
Jul 13 19:53:15.392323 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:15.392283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:15.544459 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:15.544423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:19.106673 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:19.106639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:53:21.720508 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.720470    2562 patch_prober.go:28] interesting pod/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 13 19:53:21.720948 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.720527    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 13 19:53:21.791791 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.791761    2562 generic.go:356] "Generic (PLEG): container finished" podID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerID="36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad" exitCode=0
Jul 13 19:53:21.791933 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.791827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" event={"ID":"9d43a86f-7615-4134-a476-a6dcd11ba3e3","Type":"ContainerDied","Data":"36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"}
Jul 13 19:53:21.792205 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.792190    2562 scope.go:117] "RemoveContainer" containerID="36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"
Jul 13 19:53:21.793181 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.793151    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:53:21.793299 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.793278    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:53:21.793374 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:21.793346    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:53:22.799092 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:22.799057    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" event={"ID":"9d43a86f-7615-4134-a476-a6dcd11ba3e3","Type":"ContainerStarted","Data":"0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a"}
Jul 13 19:53:22.960369 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:22.960328    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:22.960614 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:22.960594    2562 patch_prober.go:28] interesting pod/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 13 19:53:22.960663 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:22.960638    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 13 19:53:23.960887 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:23.960849    2562 patch_prober.go:28] interesting pod/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 13 19:53:23.961351 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:23.960910    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 13 19:53:24.960395 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:24.960357    2562 patch_prober.go:28] interesting pod/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 13 19:53:24.960611 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:24.960420    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 13 19:53:25.961194 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:25.961153    2562 patch_prober.go:28] interesting pod/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 13 19:53:25.961651 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:25.961217    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 13 19:53:26.964073 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:26.964037    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:26.964457 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:26.964397    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:53:26.965161 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:53:26.965136    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:54:19.718249 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:19.718211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:24.336349 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:24.336308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:30.161701 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:30.161667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:30.231032 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:30.231000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:34.512200 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:34.512165    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"]
Jul 13 19:54:34.512636 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:34.512402    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy" containerID="cri-o://0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a" gracePeriod=30
Jul 13 19:54:34.540786 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:34.540758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:35.157620 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:35.157591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:35.259809 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:35.259774    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:35.359505 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:35.359472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:36.876480 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:36.876444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:37.287850 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:37.287771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:37.389117 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:37.389080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:39.764874 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.764851    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:54:39.847159 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847134    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-token\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847198    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-socket\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847234    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istiod-ca-cert\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847257    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-credential-socket\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847281    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd2k8\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-kube-api-access-xd2k8\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847328 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847302    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-data\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847617 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847330    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-envoy\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847617 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847381    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-certs\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847617 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-podinfo\") pod \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\" (UID: \"9d43a86f-7615-4134-a476-a6dcd11ba3e3\") "
Jul 13 19:54:39.847763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847644    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:54:39.847763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847738    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:54:39.847874 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847754    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.847946 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.847925    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:54:39.848428 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.848366    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:54:39.848428 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.848379    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:54:39.849845 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.849819    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:54:39.849959 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.849933    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-kube-api-access-xd2k8" (OuterVolumeSpecName: "kube-api-access-xd2k8") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "kube-api-access-xd2k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:54:39.850088 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.850065    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:54:39.850144 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.850070    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d43a86f-7615-4134-a476-a6dcd11ba3e3" (UID: "9d43a86f-7615-4134-a476-a6dcd11ba3e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:54:39.948820 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948784    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948820 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948819    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948828    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948836    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948847    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd2k8\" (UniqueName: \"kubernetes.io/projected/9d43a86f-7615-4134-a476-a6dcd11ba3e3-kube-api-access-xd2k8\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948855    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948863    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:39.948987 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:39.948870    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d43a86f-7615-4134-a476-a6dcd11ba3e3-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:54:40.142399 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.142372    2562 generic.go:356] "Generic (PLEG): container finished" podID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerID="0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a" exitCode=0
Jul 13 19:54:40.142513 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.142405    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" event={"ID":"9d43a86f-7615-4134-a476-a6dcd11ba3e3","Type":"ContainerDied","Data":"0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a"}
Jul 13 19:54:40.142513 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.142420    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"
Jul 13 19:54:40.142513 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.142440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm" event={"ID":"9d43a86f-7615-4134-a476-a6dcd11ba3e3","Type":"ContainerDied","Data":"f470a3010d9496ec14a573b329189a3f8c59e83af55e7566861f237718928d6b"}
Jul 13 19:54:40.142513 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.142460    2562 scope.go:117] "RemoveContainer" containerID="0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a"
Jul 13 19:54:40.151232 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.151214    2562 scope.go:117] "RemoveContainer" containerID="36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"
Jul 13 19:54:40.159586 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.159570    2562 scope.go:117] "RemoveContainer" containerID="0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a"
Jul 13 19:54:40.159873 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:54:40.159854    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a\": container with ID starting with 0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a not found: ID does not exist" containerID="0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a"
Jul 13 19:54:40.159924 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.159881    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a"} err="failed to get container status \"0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a\": rpc error: code = NotFound desc = could not find container \"0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a\": container with ID starting with 0a2cec570341df056ec94cdfc0d57550867ee976a0ec9f7db2f16e1ff8e3f84a not found: ID does not exist"
Jul 13 19:54:40.159924 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.159899    2562 scope.go:117] "RemoveContainer" containerID="36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"
Jul 13 19:54:40.160132 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.160112    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"]
Jul 13 19:54:40.160189 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:54:40.160131    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad\": container with ID starting with 36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad not found: ID does not exist" containerID="36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"
Jul 13 19:54:40.160189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.160159    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad"} err="failed to get container status \"36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad\": rpc error: code = NotFound desc = could not find container \"36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad\": container with ID starting with 36cdce0ec4688cb534a07a9a7e3b002b76b8cbe3f93aa59302ea57994f1798ad not found: ID does not exist"
Jul 13 19:54:40.166368 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:40.166350    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7a28704c-openshift-default-c44d9b747-xcbkm"]
Jul 13 19:54:41.115358 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:41.115322    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:42.091048 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:42.091012    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" path="/var/lib/kubelet/pods/9d43a86f-7615-4134-a476-a6dcd11ba3e3/volumes"
Jul 13 19:54:42.642518 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:42.642480    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:42.716114 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:42.716087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:42.803203 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:42.803171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:42.901951 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:42.901867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:54:43.392972 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:54:43.392941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:09.368116 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:09.368079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:14.972243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:14.972213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:15.087148 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:15.087116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:15.180222 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:15.180189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:31.096635 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:31.096580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:36.845732 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:36.845685    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:36.907876 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:36.907841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:36.995463 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:36.995432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:37.094480 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:37.094427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:47.600766 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:47.600717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:48.441988 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:48.441952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:54.615552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:54.615489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:54.682633 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:54.682600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:54.763608 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:54.763581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:54.864706 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:54.864672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:54.964281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:54.964212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:55.687344 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:55.687310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:55.791607 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:55.791575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:55.893687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:55.893654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:55:57.511678 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:55:57.511649    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:03.287015 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:03.286975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:03.391453 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:03.391415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:03.961000 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:03.960965    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:10.424168 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:10.424097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:10.534760 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:10.534727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:10.689223 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:10.689144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:11.028501 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028426    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"]
Jul 13 19:56:11.028796 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028782    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy"
Jul 13 19:56:11.028840 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028798    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy"
Jul 13 19:56:11.028840 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028814    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy"
Jul 13 19:56:11.028840 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028819    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy"
Jul 13 19:56:11.028840 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028829    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy"
Jul 13 19:56:11.028840 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028834    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy"
Jul 13 19:56:11.028989 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028887    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy"
Jul 13 19:56:11.028989 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028897    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy"
Jul 13 19:56:11.028989 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.028906    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy"
Jul 13 19:56:11.031841 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.031823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.035745 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.035728    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ccb43b5e-openshift-default-dockercfg-mr766\""
Jul 13 19:56:11.087090 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.087057    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"]
Jul 13 19:56:11.138161 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.138129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:11.177196 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-certs\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177338 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280b5ae9-252f-42a5-b809-310940eef0b2-istio-podinfo\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177338 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-istio-token\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177338 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92phs\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-kube-api-access-92phs\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177338 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-envoy\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177581 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-socket\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177581 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-credential-socket\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177581 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177436    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-data\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.177581 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.177449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280b5ae9-252f-42a5-b809-310940eef0b2-istiod-ca-cert\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.277997 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.277968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-credential-socket\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-data\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280b5ae9-252f-42a5-b809-310940eef0b2-istiod-ca-cert\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-certs\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280b5ae9-252f-42a5-b809-310940eef0b2-istio-podinfo\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-istio-token\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278169 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92phs\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-kube-api-access-92phs\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278649 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-envoy\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278649 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-socket\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278649 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-data\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278783 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278669    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-socket\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278783 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-credential-socket\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.278866 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-certs\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.279087 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.278974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280b5ae9-252f-42a5-b809-310940eef0b2-istiod-ca-cert\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.280630 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.280610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-envoy\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.280778 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.280761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280b5ae9-252f-42a5-b809-310940eef0b2-istio-podinfo\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.288183 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.288159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92phs\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-kube-api-access-92phs\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.288250 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.288167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-istio-token\") pod \"e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") " pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.341730 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.341709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:11.459356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.459327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:11.479590 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.479565    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"]
Jul 13 19:56:11.481523 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:56:11.481495    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280b5ae9_252f_42a5_b809_310940eef0b2.slice/crio-7a4e734cb495b9b84930c3bd88b66a5ee2fef1af7bc8a413735ccd6303681c1c WatchSource:0}: Error finding container 7a4e734cb495b9b84930c3bd88b66a5ee2fef1af7bc8a413735ccd6303681c1c: Status 404 returned error can't find the container with id 7a4e734cb495b9b84930c3bd88b66a5ee2fef1af7bc8a413735ccd6303681c1c
Jul 13 19:56:11.483673 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.483644    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:56:11.483743 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.483707    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:56:11.483743 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.483733    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:56:11.527135 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:11.527098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" event={"ID":"280b5ae9-252f-42a5-b809-310940eef0b2","Type":"ContainerStarted","Data":"7a4e734cb495b9b84930c3bd88b66a5ee2fef1af7bc8a413735ccd6303681c1c"}
Jul 13 19:56:12.532834 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:12.532798    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" event={"ID":"280b5ae9-252f-42a5-b809-310940eef0b2","Type":"ContainerStarted","Data":"86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"}
Jul 13 19:56:12.564847 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:12.564802    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podStartSLOduration=1.564787404 podStartE2EDuration="1.564787404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:56:12.563942129 +0000 UTC m=+2068.979218323" watchObservedRunningTime="2026-07-13 19:56:12.564787404 +0000 UTC m=+2068.980063584"
Jul 13 19:56:13.342734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:13.342702    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:13.347292 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:13.347269    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:13.537063 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:13.537036    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:13.538053 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:13.538034    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:14.630471 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.630434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:14.725778 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.725740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:14.726807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.726785    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"]
Jul 13 19:56:14.727131 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.727117    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy"
Jul 13 19:56:14.727189 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.727133    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9421b4b8-fe73-489e-8a5b-81c01cae23ab" containerName="istio-proxy"
Jul 13 19:56:14.727224 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.727197    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d43a86f-7615-4134-a476-a6dcd11ba3e3" containerName="istio-proxy"
Jul 13 19:56:14.730082 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.730067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:14.732763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.732737    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ccb43b5e-serving-cert\""
Jul 13 19:56:14.732984 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.732965    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\""
Jul 13 19:56:14.743140 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.743118    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"]
Jul 13 19:56:14.809792 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.809764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:14.909629 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.909522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-openshift-service-ca\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:14.909629 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.909593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-maas-api-tls\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:14.909808 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.909685    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2bh\" (UniqueName: \"kubernetes.io/projected/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-kube-api-access-wx2bh\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:14.949102 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:14.949077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:15.010261 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.010225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-openshift-service-ca\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.010417 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.010266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-maas-api-tls\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.010417 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.010316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2bh\" (UniqueName: \"kubernetes.io/projected/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-kube-api-access-wx2bh\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.010892 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.010875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-openshift-service-ca\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.013000 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.012983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-maas-api-tls\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.020202 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.020180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2bh\" (UniqueName: \"kubernetes.io/projected/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-kube-api-access-wx2bh\") pod \"maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.041044 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.041021    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:15.174809 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.174786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"]
Jul 13 19:56:15.177022 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:56:15.176990    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a1f70d8_bda5_4f6a_a8be_b6fc13889109.slice/crio-03f91786d0da6602337af9e4278edcdb907d09eca4b20d308927e95725199497 WatchSource:0}: Error finding container 03f91786d0da6602337af9e4278edcdb907d09eca4b20d308927e95725199497: Status 404 returned error can't find the container with id 03f91786d0da6602337af9e4278edcdb907d09eca4b20d308927e95725199497
Jul 13 19:56:15.178452 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.178435    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:56:15.306402 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.306356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:15.407274 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.407231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:15.547224 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:15.547131    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" event={"ID":"2a1f70d8-bda5-4f6a-a8be-b6fc13889109","Type":"ContainerStarted","Data":"03f91786d0da6602337af9e4278edcdb907d09eca4b20d308927e95725199497"}
Jul 13 19:56:16.552680 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:16.552640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" event={"ID":"2a1f70d8-bda5-4f6a-a8be-b6fc13889109","Type":"ContainerStarted","Data":"74c7bd491e1aac844d4e0a7292e5a41e44f20d0ab44497c2cb77078a092c1358"}
Jul 13 19:56:16.553049 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:16.552747    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:16.573402 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:16.573353    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" podStartSLOduration=2.091075953 podStartE2EDuration="2.573340935s" totalImagesPullingTime="482.264982ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:56:16.571143399 +0000 UTC m=+2072.986419581" watchObservedRunningTime="2026-07-13 19:56:16.573340935 +0000 UTC m=+2072.988617117"
Jul 13 19:56:19.161226 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:19.161136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:19.828356 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:19.828323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:19.928298 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:19.928265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:20.271371 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:20.271280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:22.563182 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:22.563155    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:26.291443 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:26.291405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:26.356239 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:26.356200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:28.538698 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.538665    2562 patch_prober.go:28] interesting pod/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 13 19:56:28.539074 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.538720    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 13 19:56:28.607281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.607254    2562 generic.go:356] "Generic (PLEG): container finished" podID="280b5ae9-252f-42a5-b809-310940eef0b2" containerID="86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd" exitCode=0
Jul 13 19:56:28.607428 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.607329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" event={"ID":"280b5ae9-252f-42a5-b809-310940eef0b2","Type":"ContainerDied","Data":"86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"}
Jul 13 19:56:28.607763 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.607744    2562 scope.go:117] "RemoveContainer" containerID="86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"
Jul 13 19:56:28.608663 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.608638    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:56:28.608770 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.608698    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:56:28.608770 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:28.608728    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:56:29.613790 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:29.613751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" event={"ID":"280b5ae9-252f-42a5-b809-310940eef0b2","Type":"ContainerStarted","Data":"7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc"}
Jul 13 19:56:30.342701 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:30.342665    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:30.342938 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:30.342911    2562 patch_prober.go:28] interesting pod/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 13 19:56:30.343016 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:30.342971    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 13 19:56:31.342591 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:31.342555    2562 patch_prober.go:28] interesting pod/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 13 19:56:31.343154 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:31.343120    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 13 19:56:32.342170 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:32.342127    2562 patch_prober.go:28] interesting pod/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 13 19:56:32.342370 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:32.342200    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 13 19:56:33.343191 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:33.343154    2562 patch_prober.go:28] interesting pod/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 13 19:56:33.343679 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:33.343217    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 13 19:56:34.346021 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:34.345994    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:34.346484 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:34.346265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:34.346931 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:34.346917    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:36.048734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:36.048700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:36.252286 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:36.252250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:37.200891 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.200865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:37.301920 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.301867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:37.441181 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.441149    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"]
Jul 13 19:56:37.441458 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.441433    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" podUID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" containerName="maas-api" containerID="cri-o://74c7bd491e1aac844d4e0a7292e5a41e44f20d0ab44497c2cb77078a092c1358" gracePeriod=30
Jul 13 19:56:37.467915 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.467890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:37.558128 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.558100    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" podUID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.40:8443/health\": dial tcp 10.133.0.40:8443: connect: connection refused"
Jul 13 19:56:37.657841 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.657812    2562 generic.go:356] "Generic (PLEG): container finished" podID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" containerID="74c7bd491e1aac844d4e0a7292e5a41e44f20d0ab44497c2cb77078a092c1358" exitCode=0
Jul 13 19:56:37.657994 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.657880    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" event={"ID":"2a1f70d8-bda5-4f6a-a8be-b6fc13889109","Type":"ContainerDied","Data":"74c7bd491e1aac844d4e0a7292e5a41e44f20d0ab44497c2cb77078a092c1358"}
Jul 13 19:56:37.692508 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.692487    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:37.800522 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.800445    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-maas-api-tls\") pod \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") "
Jul 13 19:56:37.800522 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.800496    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx2bh\" (UniqueName: \"kubernetes.io/projected/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-kube-api-access-wx2bh\") pod \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") "
Jul 13 19:56:37.800750 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.800578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-openshift-service-ca\") pod \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\" (UID: \"2a1f70d8-bda5-4f6a-a8be-b6fc13889109\") "
Jul 13 19:56:37.800860 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.800832    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a1f70d8-bda5-4f6a-a8be-b6fc13889109" (UID: "2a1f70d8-bda5-4f6a-a8be-b6fc13889109"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:56:37.802624 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.802605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a1f70d8-bda5-4f6a-a8be-b6fc13889109" (UID: "2a1f70d8-bda5-4f6a-a8be-b6fc13889109"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:56:37.802700 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.802632    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-kube-api-access-wx2bh" (OuterVolumeSpecName: "kube-api-access-wx2bh") pod "2a1f70d8-bda5-4f6a-a8be-b6fc13889109" (UID: "2a1f70d8-bda5-4f6a-a8be-b6fc13889109"). InnerVolumeSpecName "kube-api-access-wx2bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:56:37.901443 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.901403    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-openshift-service-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:37.901443 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.901438    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-maas-api-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:37.901443 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:37.901447    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx2bh\" (UniqueName: \"kubernetes.io/projected/2a1f70d8-bda5-4f6a-a8be-b6fc13889109-kube-api-access-wx2bh\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:38.338805 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.338769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:38.529301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.529215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:38.662899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.662872    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"
Jul 13 19:56:38.662899 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.662876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2" event={"ID":"2a1f70d8-bda5-4f6a-a8be-b6fc13889109","Type":"ContainerDied","Data":"03f91786d0da6602337af9e4278edcdb907d09eca4b20d308927e95725199497"}
Jul 13 19:56:38.663134 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.662963    2562 scope.go:117] "RemoveContainer" containerID="74c7bd491e1aac844d4e0a7292e5a41e44f20d0ab44497c2cb77078a092c1358"
Jul 13 19:56:38.684703 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.684679    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"]
Jul 13 19:56:38.689589 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:38.689569    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccb43b5e-789d5bfd58-ktvm2"]
Jul 13 19:56:40.090749 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:40.090716    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" path="/var/lib/kubelet/pods/2a1f70d8-bda5-4f6a-a8be-b6fc13889109/volumes"
Jul 13 19:56:41.742827 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:41.742794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:47.550116 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:47.550079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:47.616874 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:47.616845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:52.412091 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:52.412056    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"]
Jul 13 19:56:52.412567 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:52.412262    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy" containerID="cri-o://7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc" gracePeriod=30
Jul 13 19:56:52.434800 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:52.434773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:52.662771 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:52.662689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:52.796142 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:52.796110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:52.892472 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:52.892444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:53.016900 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:53.016812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:57.666251 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.666230    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:57.745369 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.745291    2562 generic.go:356] "Generic (PLEG): container finished" podID="280b5ae9-252f-42a5-b809-310940eef0b2" containerID="7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc" exitCode=0
Jul 13 19:56:57.745369 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.745353    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"
Jul 13 19:56:57.745560 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.745374    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" event={"ID":"280b5ae9-252f-42a5-b809-310940eef0b2","Type":"ContainerDied","Data":"7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc"}
Jul 13 19:56:57.745560 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.745422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt" event={"ID":"280b5ae9-252f-42a5-b809-310940eef0b2","Type":"ContainerDied","Data":"7a4e734cb495b9b84930c3bd88b66a5ee2fef1af7bc8a413735ccd6303681c1c"}
Jul 13 19:56:57.745560 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.745445    2562 scope.go:117] "RemoveContainer" containerID="7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc"
Jul 13 19:56:57.755572 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.755526    2562 scope.go:117] "RemoveContainer" containerID="86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"
Jul 13 19:56:57.759718 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.759696    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92phs\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-kube-api-access-92phs\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.759807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.759738    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-data\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.759807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.759766    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280b5ae9-252f-42a5-b809-310940eef0b2-istiod-ca-cert\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.759807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.759797    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-socket\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.759998 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.759977    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-certs\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.760048 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760031    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-envoy\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.760048 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760038    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:56:57.760141 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760083    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280b5ae9-252f-42a5-b809-310940eef0b2-istio-podinfo\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.760141 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760107    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-istio-token\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.760245 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760175    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-credential-socket\") pod \"280b5ae9-252f-42a5-b809-310940eef0b2\" (UID: \"280b5ae9-252f-42a5-b809-310940eef0b2\") "
Jul 13 19:56:57.760299 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:56:57.760529 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760460    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:56:57.760529 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760477    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:56:57.760529 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760567    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/280b5ae9-252f-42a5-b809-310940eef0b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:56:57.760529 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760575    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.760944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760611    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.760944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760627    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.760944 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.760639    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.762295 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.762262    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-kube-api-access-92phs" (OuterVolumeSpecName: "kube-api-access-92phs") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "kube-api-access-92phs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:56:57.762435 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.762375    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/280b5ae9-252f-42a5-b809-310940eef0b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:56:57.762710 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.762676    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:56:57.762799 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.762715    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "280b5ae9-252f-42a5-b809-310940eef0b2" (UID: "280b5ae9-252f-42a5-b809-310940eef0b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:56:57.763632 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.763614    2562 scope.go:117] "RemoveContainer" containerID="7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc"
Jul 13 19:56:57.763909 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:56:57.763891    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc\": container with ID starting with 7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc not found: ID does not exist" containerID="7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc"
Jul 13 19:56:57.763975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.763917    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc"} err="failed to get container status \"7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc\": rpc error: code = NotFound desc = could not find container \"7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc\": container with ID starting with 7bd05bbeee05582d9f5f504463baffc6eea30b58f0ff3c6078db383257a2d5cc not found: ID does not exist"
Jul 13 19:56:57.763975 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.763936    2562 scope.go:117] "RemoveContainer" containerID="86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"
Jul 13 19:56:57.764166 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:56:57.764152    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd\": container with ID starting with 86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd not found: ID does not exist" containerID="86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"
Jul 13 19:56:57.764221 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.764169    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd"} err="failed to get container status \"86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd\": rpc error: code = NotFound desc = could not find container \"86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd\": container with ID starting with 86a44ddb6767d7b288e1a5257bb8d4655bc9cf1d67df37258bbf63f2423861dd not found: ID does not exist"
Jul 13 19:56:57.861278 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.861254    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92phs\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-kube-api-access-92phs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.861278 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.861276    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280b5ae9-252f-42a5-b809-310940eef0b2-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.861423 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.861285    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280b5ae9-252f-42a5-b809-310940eef0b2-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.861423 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.861293    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280b5ae9-252f-42a5-b809-310940eef0b2-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:57.861423 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:57.861301    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280b5ae9-252f-42a5-b809-310940eef0b2-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:56:58.066434 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:58.066409    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"]
Jul 13 19:56:58.074925 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:58.071567    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ccb43b5e-openshift-default-5c585c4fbc-fdfgt"]
Jul 13 19:56:58.090505 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:58.090481    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" path="/var/lib/kubelet/pods/280b5ae9-252f-42a5-b809-310940eef0b2/volumes"
Jul 13 19:56:58.918887 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:58.918823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:58.983760 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:58.983718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:59.113450 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:59.113411    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:59.213451 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:59.213372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:56:59.313333 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:56:59.313293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:05.219558 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:05.219510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:05.319500 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:05.319463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:12.100059 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.100024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:12.421200 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421121    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"]
Jul 13 19:57:12.421561 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421519    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy"
Jul 13 19:57:12.421561 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421555    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy"
Jul 13 19:57:12.421704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421575    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy"
Jul 13 19:57:12.421704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421584    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy"
Jul 13 19:57:12.421704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421594    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" containerName="maas-api"
Jul 13 19:57:12.421704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421602    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" containerName="maas-api"
Jul 13 19:57:12.421704 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421704    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy"
Jul 13 19:57:12.421938 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.421718    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a1f70d8-bda5-4f6a-a8be-b6fc13889109" containerName="maas-api"
Jul 13 19:57:12.426045 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.426024    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.428734 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.428714    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d67d76ca-openshift-default-dockercfg-svxsn\""
Jul 13 19:57:12.440429 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.440401    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"]
Jul 13 19:57:12.441131 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.441095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:12.581097 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-data\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-certs\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-podinfo\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-token\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d728ae82-df69-4df2-853b-6e5236fe5e7c-istiod-ca-cert\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-credential-socket\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6mq\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-kube-api-access-kb6mq\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581497 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-envoy\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.581497 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.581365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-socket\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682501 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-data\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682501 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-certs\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-podinfo\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-token\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d728ae82-df69-4df2-853b-6e5236fe5e7c-istiod-ca-cert\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-credential-socket\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6mq\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-kube-api-access-kb6mq\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-envoy\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.682807 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-socket\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.683139 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-data\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.683139 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.682926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-certs\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.683139 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.683058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-credential-socket\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.683139 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.683102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-socket\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.683357 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.683339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d728ae82-df69-4df2-853b-6e5236fe5e7c-istiod-ca-cert\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.685093 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.685076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-envoy\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.685408 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.685387    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-podinfo\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.695432 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.695407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-token\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.703690 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.703665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6mq\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-kube-api-access-kb6mq\") pod \"e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") " pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.741059 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.741037    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:12.880762 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.880734    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"]
Jul 13 19:57:12.882483 ip-10-0-137-89 kubenswrapper[2562]: W0713 19:57:12.882453    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd728ae82_df69_4df2_853b_6e5236fe5e7c.slice/crio-40aead83648233c1aa2bff6ff9e4ac6358f8da757caa8af06a6ffbda4d69df32 WatchSource:0}: Error finding container 40aead83648233c1aa2bff6ff9e4ac6358f8da757caa8af06a6ffbda4d69df32: Status 404 returned error can't find the container with id 40aead83648233c1aa2bff6ff9e4ac6358f8da757caa8af06a6ffbda4d69df32
Jul 13 19:57:12.884491 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.884462    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:57:12.884565 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.884526    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:57:12.884607 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.884580    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:57:12.936982 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:12.936958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:13.816311 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:13.816268    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" event={"ID":"d728ae82-df69-4df2-853b-6e5236fe5e7c","Type":"ContainerStarted","Data":"8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"}
Jul 13 19:57:13.816805 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:13.816326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" event={"ID":"d728ae82-df69-4df2-853b-6e5236fe5e7c","Type":"ContainerStarted","Data":"40aead83648233c1aa2bff6ff9e4ac6358f8da757caa8af06a6ffbda4d69df32"}
Jul 13 19:57:13.836155 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:13.836104    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" podStartSLOduration=1.836090056 podStartE2EDuration="1.836090056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:57:13.83508673 +0000 UTC m=+2130.250362912" watchObservedRunningTime="2026-07-13 19:57:13.836090056 +0000 UTC m=+2130.251366238"
Jul 13 19:57:14.741365 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:14.741335    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:14.746300 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:14.746274    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:14.820340 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:14.820318    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:14.821255 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:14.821240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:15.746710 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:15.746675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:15.847176 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:15.847144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:15.918160 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:15.918133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:16.210504 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:16.210467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:16.518042 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:16.517944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:16.619738 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:16.619708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:20.261841 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:20.261807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:29.231077 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.231039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:57:29.822276 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.822238    2562 patch_prober.go:28] interesting pod/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 13 19:57:29.822455 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.822292    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 13 19:57:29.894301 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.894275    2562 generic.go:356] "Generic (PLEG): container finished" podID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerID="8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8" exitCode=0
Jul 13 19:57:29.894462 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.894349    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" event={"ID":"d728ae82-df69-4df2-853b-6e5236fe5e7c","Type":"ContainerDied","Data":"8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"}
Jul 13 19:57:29.894753 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.894740    2562 scope.go:117] "RemoveContainer" containerID="8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"
Jul 13 19:57:29.895765 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.895738    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:57:29.895843 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.895827    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:57:29.895885 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:29.895869    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 19:57:30.900758 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:30.900722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" event={"ID":"d728ae82-df69-4df2-853b-6e5236fe5e7c","Type":"ContainerStarted","Data":"bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86"}
Jul 13 19:57:31.741914 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:31.741877    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:31.742290 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:31.742200    2562 patch_prober.go:28] interesting pod/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 13 19:57:31.742457 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:31.742434    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 13 19:57:32.741932 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:32.741885    2562 patch_prober.go:28] interesting pod/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 13 19:57:32.742400 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:32.741954    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 13 19:57:33.742040 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:33.741992    2562 patch_prober.go:28] interesting pod/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 13 19:57:33.742552 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:33.742062    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 13 19:57:34.745901 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:34.745817    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:34.746326 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:34.746195    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:57:34.747005 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:57:34.746979    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:58:14.659733 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:58:14.659681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:58:19.097973 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:58:19.097930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:58:24.171075 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:58:24.171041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:00.219411 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:00.219375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:01.180167 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:01.180140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:06.901531 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:06.901445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:12.684327 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:12.684284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:12.775231 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:12.775201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:21.398029 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:21.397999    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"]
Jul 13 19:59:21.398394 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:21.398272    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy" containerID="cri-o://bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86" gracePeriod=30
Jul 13 19:59:21.413208 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:21.413170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:24.122068 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:24.122035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:25.359331 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:25.359299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:26.645335 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.645309    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:59:26.807216 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807192    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-credential-socket\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807234    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-data\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807269    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d728ae82-df69-4df2-853b-6e5236fe5e7c-istiod-ca-cert\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807314    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-token\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807339    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-envoy\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807382 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-podinfo\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807406    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-socket\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807468    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb6mq\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-kube-api-access-kb6mq\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807501    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-certs\") pod \"d728ae82-df69-4df2-853b-6e5236fe5e7c\" (UID: \"d728ae82-df69-4df2-853b-6e5236fe5e7c\") "
Jul 13 19:59:26.807687 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807511    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:59:26.807884 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807767    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.807884 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807780    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d728ae82-df69-4df2-853b-6e5236fe5e7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:59:26.808016 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.807983    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:59:26.808016 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.808008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:59:26.808587 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.808562    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:59:26.809677 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.809654    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:59:26.809913 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.809893    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:59:26.810218 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.810194    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-kube-api-access-kb6mq" (OuterVolumeSpecName: "kube-api-access-kb6mq") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "kube-api-access-kb6mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:59:26.810281 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.810217    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "d728ae82-df69-4df2-853b-6e5236fe5e7c" (UID: "d728ae82-df69-4df2-853b-6e5236fe5e7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:59:26.909072 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909039    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb6mq\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-kube-api-access-kb6mq\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909072 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909070    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909072 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909079    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909087    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d728ae82-df69-4df2-853b-6e5236fe5e7c-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909095    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909102    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909109    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d728ae82-df69-4df2-853b-6e5236fe5e7c-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:26.909243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:26.909117    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d728ae82-df69-4df2-853b-6e5236fe5e7c-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 19:59:27.390239 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.390210    2562 generic.go:356] "Generic (PLEG): container finished" podID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerID="bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86" exitCode=0
Jul 13 19:59:27.390400 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.390266    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"
Jul 13 19:59:27.390400 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.390280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" event={"ID":"d728ae82-df69-4df2-853b-6e5236fe5e7c","Type":"ContainerDied","Data":"bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86"}
Jul 13 19:59:27.390400 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.390324    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6" event={"ID":"d728ae82-df69-4df2-853b-6e5236fe5e7c","Type":"ContainerDied","Data":"40aead83648233c1aa2bff6ff9e4ac6358f8da757caa8af06a6ffbda4d69df32"}
Jul 13 19:59:27.390400 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.390346    2562 scope.go:117] "RemoveContainer" containerID="bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86"
Jul 13 19:59:27.398886 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.398870    2562 scope.go:117] "RemoveContainer" containerID="8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"
Jul 13 19:59:27.406386 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.406371    2562 scope.go:117] "RemoveContainer" containerID="bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86"
Jul 13 19:59:27.406661 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:59:27.406637    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86\": container with ID starting with bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86 not found: ID does not exist" containerID="bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86"
Jul 13 19:59:27.406736 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.406672    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86"} err="failed to get container status \"bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86\": rpc error: code = NotFound desc = could not find container \"bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86\": container with ID starting with bcd1f63a68fa43c60e8ba04d0fd1d560afcd8ea5f5597cd26f39f0333c193b86 not found: ID does not exist"
Jul 13 19:59:27.406736 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.406691    2562 scope.go:117] "RemoveContainer" containerID="8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"
Jul 13 19:59:27.406941 ip-10-0-137-89 kubenswrapper[2562]: E0713 19:59:27.406926    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8\": container with ID starting with 8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8 not found: ID does not exist" containerID="8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"
Jul 13 19:59:27.406983 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.406945    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8"} err="failed to get container status \"8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8\": rpc error: code = NotFound desc = could not find container \"8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8\": container with ID starting with 8e5a2351d1b01a7191ac9e475190863b050b222f304c4a6c0b5f675dffb619c8 not found: ID does not exist"
Jul 13 19:59:27.415516 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.415494    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"]
Jul 13 19:59:27.421606 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:27.421585    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d67d76ca-openshift-default-778b44855-w8pd6"]
Jul 13 19:59:28.090471 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:28.090441    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" path="/var/lib/kubelet/pods/d728ae82-df69-4df2-853b-6e5236fe5e7c/volumes"
Jul 13 19:59:31.628708 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:31.628672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:31.728995 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:31.728959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:32.343243 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:32.343210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:38.237147 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:38.237113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:38.358716 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:38.358672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:38.447097 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:38.447066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:48.960437 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:48.960394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 19:59:49.380006 ip-10-0-137-89 kubenswrapper[2562]: I0713 19:59:49.379973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:10.310550 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:10.310501    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:17.147073 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147044    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"]
Jul 13 20:00:17.147429 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147354    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy"
Jul 13 20:00:17.147429 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147364    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy"
Jul 13 20:00:17.147429 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147377    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy"
Jul 13 20:00:17.147429 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147383    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy"
Jul 13 20:00:17.147581 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147440    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="280b5ae9-252f-42a5-b809-310940eef0b2" containerName="istio-proxy"
Jul 13 20:00:17.147581 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147449    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy"
Jul 13 20:00:17.147581 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.147455    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d728ae82-df69-4df2-853b-6e5236fe5e7c" containerName="istio-proxy"
Jul 13 20:00:17.150381 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.150363    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.152990 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.152970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-dffe11-openshift-default-dockercfg-k5h4c\""
Jul 13 20:00:17.168828 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.168806    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"]
Jul 13 20:00:17.310496 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-data\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310689 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzm2x\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-kube-api-access-nzm2x\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310689 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-istio-token\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310689 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-envoy\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310689 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-certs\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310689 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310627    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-socket\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310689 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e42a1a-66e6-4552-a0f1-826977228216-istio-podinfo\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310891 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e42a1a-66e6-4552-a0f1-826977228216-istiod-ca-cert\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.310891 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.310717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-credential-socket\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411183 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-envoy\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411183 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-certs\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411183 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-socket\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411414 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e42a1a-66e6-4552-a0f1-826977228216-istio-podinfo\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411414 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e42a1a-66e6-4552-a0f1-826977228216-istiod-ca-cert\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411414 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-credential-socket\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411414 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411374    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-data\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411414 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzm2x\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-kube-api-access-nzm2x\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411695 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-istio-token\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411695 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-certs\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411695 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-socket\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411858 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-credential-socket\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.411912 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.411881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-data\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.412034 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.412010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e42a1a-66e6-4552-a0f1-826977228216-istiod-ca-cert\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.413501 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.413481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:17.414341 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.414319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-envoy\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.414559 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.414523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e42a1a-66e6-4552-a0f1-826977228216-istio-podinfo\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.420402 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.420365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzm2x\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-kube-api-access-nzm2x\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.420737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.420720    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-istio-token\") pod \"e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") " pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.460006 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.459982    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:17.593929 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.593905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"]
Jul 13 20:00:17.596819 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:00:17.596792    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e42a1a_66e6_4552_a0f1_826977228216.slice/crio-cd9121c38d3e767cd0b0bf6e40e94f3155058658ac990b0e889c17c7cc8bb4a8 WatchSource:0}: Error finding container cd9121c38d3e767cd0b0bf6e40e94f3155058658ac990b0e889c17c7cc8bb4a8: Status 404 returned error can't find the container with id cd9121c38d3e767cd0b0bf6e40e94f3155058658ac990b0e889c17c7cc8bb4a8
Jul 13 20:00:17.599187 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.599162    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:00:17.599297 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.599223    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:00:17.599297 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.599255    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:00:17.612827 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:17.612805    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" event={"ID":"67e42a1a-66e6-4552-a0f1-826977228216","Type":"ContainerStarted","Data":"cd9121c38d3e767cd0b0bf6e40e94f3155058658ac990b0e889c17c7cc8bb4a8"}
Jul 13 20:00:18.367142 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:18.367106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:18.467954 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:18.467923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:18.619232 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:18.619134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" event={"ID":"67e42a1a-66e6-4552-a0f1-826977228216","Type":"ContainerStarted","Data":"8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"}
Jul 13 20:00:18.641668 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:18.641620    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podStartSLOduration=1.641606585 podStartE2EDuration="1.641606585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:18.640429177 +0000 UTC m=+2315.055705360" watchObservedRunningTime="2026-07-13 20:00:18.641606585 +0000 UTC m=+2315.056882767"
Jul 13 20:00:19.460374 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:19.460333    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:19.465519 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:19.465494    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:19.624237 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:19.624208    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:19.625062 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:19.625045    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:20.675964 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.675915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:20.757711 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.757682    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"]
Jul 13 20:00:20.760983 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.760953    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:20.763603 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.763580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-dffe11-serving-cert\""
Jul 13 20:00:20.763716 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.763673    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\""
Jul 13 20:00:20.774987 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.774960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:20.775130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.775112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"]
Jul 13 20:00:20.867691 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.867656    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:20.941130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.941033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:20.941130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.941096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a662bb-c254-4ccf-a33c-2324f713e786-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:20.941341 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:20.941167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd5tq\" (UniqueName: \"kubernetes.io/projected/18a662bb-c254-4ccf-a33c-2324f713e786-kube-api-access-qd5tq\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.042310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.042268    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.042471 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.042328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a662bb-c254-4ccf-a33c-2324f713e786-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.042471 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.042356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd5tq\" (UniqueName: \"kubernetes.io/projected/18a662bb-c254-4ccf-a33c-2324f713e786-kube-api-access-qd5tq\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.042471 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:00:21.042428    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-serving-cert: secret "maas-api-e2e-inf-a-dffe11-serving-cert" not found
Jul 13 20:00:21.042642 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:00:21.042493    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls podName:18a662bb-c254-4ccf-a33c-2324f713e786 nodeName:}" failed. No retries permitted until 2026-07-13 20:00:21.542476558 +0000 UTC m=+2317.957752718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls") pod "maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" (UID: "18a662bb-c254-4ccf-a33c-2324f713e786") : secret "maas-api-e2e-inf-a-dffe11-serving-cert" not found
Jul 13 20:00:21.043049 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.043029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a662bb-c254-4ccf-a33c-2324f713e786-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.053581 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.053562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd5tq\" (UniqueName: \"kubernetes.io/projected/18a662bb-c254-4ccf-a33c-2324f713e786-kube-api-access-qd5tq\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.459400 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.459366    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:21.546509 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.546477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.549061 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.549039    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls\") pod \"maas-api-e2e-inf-a-dffe11-849999cb7d-x645z\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.556040 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.556003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:21.674318 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.674285    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:21.817198 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:21.817138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"]
Jul 13 20:00:21.819467 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:00:21.819430    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a662bb_c254_4ccf_a33c_2324f713e786.slice/crio-3e2f6a130d874564d904dbd8e3e8f3493730ba329a30b5ec1a276075173d0a15 WatchSource:0}: Error finding container 3e2f6a130d874564d904dbd8e3e8f3493730ba329a30b5ec1a276075173d0a15: Status 404 returned error can't find the container with id 3e2f6a130d874564d904dbd8e3e8f3493730ba329a30b5ec1a276075173d0a15
Jul 13 20:00:22.639914 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:22.639875    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" event={"ID":"18a662bb-c254-4ccf-a33c-2324f713e786","Type":"ContainerStarted","Data":"3e2f6a130d874564d904dbd8e3e8f3493730ba329a30b5ec1a276075173d0a15"}
Jul 13 20:00:23.645667 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:23.645634    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" event={"ID":"18a662bb-c254-4ccf-a33c-2324f713e786","Type":"ContainerStarted","Data":"fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8"}
Jul 13 20:00:23.646054 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:23.645767    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:23.664580 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:23.664505    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" podStartSLOduration=2.010789973 podStartE2EDuration="3.664487894s" totalImagesPullingTime="1.653697921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:23.663111162 +0000 UTC m=+2320.078387345" watchObservedRunningTime="2026-07-13 20:00:23.664487894 +0000 UTC m=+2320.079764125"
Jul 13 20:00:26.062384 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:26.062347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:26.163438 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:26.163389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:26.520951 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:26.520874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:28.551496 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.551462    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:28.620858 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.620826    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"]
Jul 13 20:00:28.624226 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.624203    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.626983 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.626955    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-7b8c26-serving-cert\""
Jul 13 20:00:28.635686 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.635663    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"]
Jul 13 20:00:28.642916 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.642895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:28.716753 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.716727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:28.813290 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.813259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e0daaf1-f29c-452c-8cfc-8f4d73137881-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.813461 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.813296    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e0daaf1-f29c-452c-8cfc-8f4d73137881-maas-api-tls\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.813461 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.813402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7kn\" (UniqueName: \"kubernetes.io/projected/1e0daaf1-f29c-452c-8cfc-8f4d73137881-kube-api-access-fz7kn\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.914562 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.914511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e0daaf1-f29c-452c-8cfc-8f4d73137881-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.914562 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.914568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e0daaf1-f29c-452c-8cfc-8f4d73137881-maas-api-tls\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.914791 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.914621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7kn\" (UniqueName: \"kubernetes.io/projected/1e0daaf1-f29c-452c-8cfc-8f4d73137881-kube-api-access-fz7kn\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.915201 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.915179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e0daaf1-f29c-452c-8cfc-8f4d73137881-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.917280 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.917253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e0daaf1-f29c-452c-8cfc-8f4d73137881-maas-api-tls\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.923070 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.923048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7kn\" (UniqueName: \"kubernetes.io/projected/1e0daaf1-f29c-452c-8cfc-8f4d73137881-kube-api-access-fz7kn\") pod \"maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:28.934722 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:28.934700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:29.064488 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.064466    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"]
Jul 13 20:00:29.067095 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:00:29.067066    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0daaf1_f29c_452c_8cfc_8f4d73137881.slice/crio-cb0f575cdad486f1ccfde3f6efdd2324bdf6c5fe800b28e66a902051cabde37e WatchSource:0}: Error finding container cb0f575cdad486f1ccfde3f6efdd2324bdf6c5fe800b28e66a902051cabde37e: Status 404 returned error can't find the container with id cb0f575cdad486f1ccfde3f6efdd2324bdf6c5fe800b28e66a902051cabde37e
Jul 13 20:00:29.655212 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.655184    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:00:29.672798 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.672739    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69" event={"ID":"1e0daaf1-f29c-452c-8cfc-8f4d73137881","Type":"ContainerStarted","Data":"59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75"}
Jul 13 20:00:29.673000 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.672803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69" event={"ID":"1e0daaf1-f29c-452c-8cfc-8f4d73137881","Type":"ContainerStarted","Data":"cb0f575cdad486f1ccfde3f6efdd2324bdf6c5fe800b28e66a902051cabde37e"}
Jul 13 20:00:29.673000 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.672879    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:29.692123 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.692066    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69" podStartSLOduration=1.287474599 podStartE2EDuration="1.692051963s" totalImagesPullingTime="404.577364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:29.689720572 +0000 UTC m=+2326.104996754" watchObservedRunningTime="2026-07-13 20:00:29.692051963 +0000 UTC m=+2326.107328150"
Jul 13 20:00:29.724195 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.724117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:29.824808 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:29.824779    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:33.067987 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:33.067944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:35.683055 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:35.682984    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:00:38.719072 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.719034    2562 generic.go:356] "Generic (PLEG): container finished" podID="67e42a1a-66e6-4552-a0f1-826977228216" containerID="8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43" exitCode=0
Jul 13 20:00:38.719463 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.719105    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" event={"ID":"67e42a1a-66e6-4552-a0f1-826977228216","Type":"ContainerDied","Data":"8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"}
Jul 13 20:00:38.719507 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.719492    2562 scope.go:117] "RemoveContainer" containerID="8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"
Jul 13 20:00:38.720489 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.720460    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:00:38.720615 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.720595    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:00:38.720662 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.720651    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:00:38.901355 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:38.901244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:39.726503 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:39.726467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" event={"ID":"67e42a1a-66e6-4552-a0f1-826977228216","Type":"ContainerStarted","Data":"fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e"}
Jul 13 20:00:40.460951 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:40.460911    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:40.461195 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:40.461171    2562 patch_prober.go:28] interesting pod/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 13 20:00:40.461273 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:40.461244    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 13 20:00:41.461247 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:41.461209    2562 patch_prober.go:28] interesting pod/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 13 20:00:41.461836 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:41.461271    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 13 20:00:42.460617 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:42.460577    2562 patch_prober.go:28] interesting pod/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 13 20:00:42.460817 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:42.460642    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 13 20:00:43.195237 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:43.195193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:43.297362 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:43.297316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:43.460947 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:43.460856    2562 patch_prober.go:28] interesting pod/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 13 20:00:43.460947 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:43.460925    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 13 20:00:44.464033 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:44.464000    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:44.464462 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:44.464262    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:44.465054 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:44.465039    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:00:50.503885 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:50.503842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:51.479728 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:51.479686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:00:58.789607 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:00:58.789572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:03.721867 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:03.721818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:14.659159 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:14.659124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:31.639466 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:31.639426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:34.351094 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:34.351058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:34.405720 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:34.405694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:37.059336 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:37.059300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:37.160354 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:37.160324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:43.381098 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.381054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:43.483025 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.482992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:43.610241 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.610209    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"]
Jul 13 20:01:43.610465 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.610425    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" podUID="18a662bb-c254-4ccf-a33c-2324f713e786" containerName="maas-api" containerID="cri-o://fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8" gracePeriod=30
Jul 13 20:01:43.637634 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.637581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:43.852259 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.852236    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:01:43.958806 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.958725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd5tq\" (UniqueName: \"kubernetes.io/projected/18a662bb-c254-4ccf-a33c-2324f713e786-kube-api-access-qd5tq\") pod \"18a662bb-c254-4ccf-a33c-2324f713e786\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") "
Jul 13 20:01:43.958806 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.958790    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a662bb-c254-4ccf-a33c-2324f713e786-openshift-service-ca\") pod \"18a662bb-c254-4ccf-a33c-2324f713e786\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") "
Jul 13 20:01:43.959006 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.958846    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls\") pod \"18a662bb-c254-4ccf-a33c-2324f713e786\" (UID: \"18a662bb-c254-4ccf-a33c-2324f713e786\") "
Jul 13 20:01:43.959165 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.959140    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a662bb-c254-4ccf-a33c-2324f713e786-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18a662bb-c254-4ccf-a33c-2324f713e786" (UID: "18a662bb-c254-4ccf-a33c-2324f713e786"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:01:43.960994 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.960966    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18a662bb-c254-4ccf-a33c-2324f713e786" (UID: "18a662bb-c254-4ccf-a33c-2324f713e786"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:01:43.961064 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:43.961034    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a662bb-c254-4ccf-a33c-2324f713e786-kube-api-access-qd5tq" (OuterVolumeSpecName: "kube-api-access-qd5tq") pod "18a662bb-c254-4ccf-a33c-2324f713e786" (UID: "18a662bb-c254-4ccf-a33c-2324f713e786"). InnerVolumeSpecName "kube-api-access-qd5tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:01:44.024868 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.024839    2562 generic.go:356] "Generic (PLEG): container finished" podID="18a662bb-c254-4ccf-a33c-2324f713e786" containerID="fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8" exitCode=0
Jul 13 20:01:44.025029 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.024882    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"
Jul 13 20:01:44.025029 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.024887    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" event={"ID":"18a662bb-c254-4ccf-a33c-2324f713e786","Type":"ContainerDied","Data":"fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8"}
Jul 13 20:01:44.025029 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.024931    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z" event={"ID":"18a662bb-c254-4ccf-a33c-2324f713e786","Type":"ContainerDied","Data":"3e2f6a130d874564d904dbd8e3e8f3493730ba329a30b5ec1a276075173d0a15"}
Jul 13 20:01:44.025029 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.024952    2562 scope.go:117] "RemoveContainer" containerID="fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8"
Jul 13 20:01:44.034509 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.034489    2562 scope.go:117] "RemoveContainer" containerID="fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8"
Jul 13 20:01:44.034824 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:01:44.034805    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8\": container with ID starting with fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8 not found: ID does not exist" containerID="fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8"
Jul 13 20:01:44.034873 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.034832    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8"} err="failed to get container status \"fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8\": rpc error: code = NotFound desc = could not find container \"fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8\": container with ID starting with fa073fc549813b2ab6f648e85f2203aff1bd128b8f09be72ae5bb5a4f2155ca8 not found: ID does not exist"
Jul 13 20:01:44.045259 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.045238    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"]
Jul 13 20:01:44.051215 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.051197    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dffe11-849999cb7d-x645z"]
Jul 13 20:01:44.059565 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.059527    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd5tq\" (UniqueName: \"kubernetes.io/projected/18a662bb-c254-4ccf-a33c-2324f713e786-kube-api-access-qd5tq\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:01:44.059633 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.059569    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a662bb-c254-4ccf-a33c-2324f713e786-openshift-service-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:01:44.059633 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.059580    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18a662bb-c254-4ccf-a33c-2324f713e786-maas-api-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:01:44.092202 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:44.092178    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a662bb-c254-4ccf-a33c-2324f713e786" path="/var/lib/kubelet/pods/18a662bb-c254-4ccf-a33c-2324f713e786/volumes"
Jul 13 20:01:47.925393 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:47.925360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:53.621806 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:53.621756    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:53.722642 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:53.722610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:58.604062 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:58.604019    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"]
Jul 13 20:01:58.604522 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:58.604216    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy" containerID="cri-o://fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e" gracePeriod=30
Jul 13 20:01:58.625670 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:58.625634    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:58.847422 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:58.847389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:01:59.467155 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:59.467109    2562 patch_prober.go:28] interesting pod/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 20:01:59.467325 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:59.467179    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 20:01:59.592725 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:01:59.592686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:00.852334 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:00.852261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:03.863392 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.863338    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:02:03.932223 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-data\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932372 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932259    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-credential-socket\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932372 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932291    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-certs\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932372 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932313    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e42a1a-66e6-4552-a0f1-826977228216-istiod-ca-cert\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932372 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932360    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-envoy\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932619 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932389    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-istio-token\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932619 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzm2x\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-kube-api-access-nzm2x\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932619 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932573    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e42a1a-66e6-4552-a0f1-826977228216-istio-podinfo\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932619 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932584    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:02:03.932813 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932620    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-socket\") pod \"67e42a1a-66e6-4552-a0f1-826977228216\" (UID: \"67e42a1a-66e6-4552-a0f1-826977228216\") "
Jul 13 20:02:03.932866 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.932813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e42a1a-66e6-4552-a0f1-826977228216-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:02:03.933052 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.933018    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:02:03.933052 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.933032    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:02:03.933178 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.933050    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:03.933178 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.933074    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e42a1a-66e6-4552-a0f1-826977228216-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:03.933306 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.933195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-data" (OuterVolumeSpecName: "istio-data") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:02:03.934775 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.934751    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-kube-api-access-nzm2x" (OuterVolumeSpecName: "kube-api-access-nzm2x") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "kube-api-access-nzm2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:02:03.934775 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.934753    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67e42a1a-66e6-4552-a0f1-826977228216-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 20:02:03.935116 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.935087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:02:03.935210 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:03.935121    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-istio-token" (OuterVolumeSpecName: "istio-token") pod "67e42a1a-66e6-4552-a0f1-826977228216" (UID: "67e42a1a-66e6-4552-a0f1-826977228216"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:02:04.033727 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033701    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e42a1a-66e6-4552-a0f1-826977228216-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.033727 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033725    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.033727 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033733    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.033957 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033741    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.033957 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033749    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e42a1a-66e6-4552-a0f1-826977228216-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.033957 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033756    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.033957 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.033763    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzm2x\" (UniqueName: \"kubernetes.io/projected/67e42a1a-66e6-4552-a0f1-826977228216-kube-api-access-nzm2x\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:04.112679 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.112651    2562 generic.go:356] "Generic (PLEG): container finished" podID="67e42a1a-66e6-4552-a0f1-826977228216" containerID="fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e" exitCode=0
Jul 13 20:02:04.112791 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.112699    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"
Jul 13 20:02:04.112791 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.112738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" event={"ID":"67e42a1a-66e6-4552-a0f1-826977228216","Type":"ContainerDied","Data":"fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e"}
Jul 13 20:02:04.112867 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.112791    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z" event={"ID":"67e42a1a-66e6-4552-a0f1-826977228216","Type":"ContainerDied","Data":"cd9121c38d3e767cd0b0bf6e40e94f3155058658ac990b0e889c17c7cc8bb4a8"}
Jul 13 20:02:04.112867 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.112822    2562 scope.go:117] "RemoveContainer" containerID="fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e"
Jul 13 20:02:04.121196 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.121169    2562 scope.go:117] "RemoveContainer" containerID="8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"
Jul 13 20:02:04.128699 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.128680    2562 scope.go:117] "RemoveContainer" containerID="fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e"
Jul 13 20:02:04.128991 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:02:04.128969    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e\": container with ID starting with fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e not found: ID does not exist" containerID="fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e"
Jul 13 20:02:04.129096 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.128998    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e"} err="failed to get container status \"fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e\": rpc error: code = NotFound desc = could not find container \"fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e\": container with ID starting with fd15fcb1f287107df42aa0633a5fbbe59ebf1b813d2ba42bd7f16b46bbe9923e not found: ID does not exist"
Jul 13 20:02:04.129096 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.129016    2562 scope.go:117] "RemoveContainer" containerID="8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"
Jul 13 20:02:04.129262 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:02:04.129244    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43\": container with ID starting with 8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43 not found: ID does not exist" containerID="8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"
Jul 13 20:02:04.129318 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.129266    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43"} err="failed to get container status \"8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43\": rpc error: code = NotFound desc = could not find container \"8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43\": container with ID starting with 8c36acd22646adac73d8c568cd04b24e2bbbbac5d946a8fbd9ca96df210bdd43 not found: ID does not exist"
Jul 13 20:02:04.129646 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.129629    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"]
Jul 13 20:02:04.133659 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:04.133639    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-dffe11-openshift-default-654c5cfb9f-b9r4z"]
Jul 13 20:02:06.091016 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:06.090941    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e42a1a-66e6-4552-a0f1-826977228216" path="/var/lib/kubelet/pods/67e42a1a-66e6-4552-a0f1-826977228216/volumes"
Jul 13 20:02:09.384189 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.384157    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"]
Jul 13 20:02:09.384645 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.384381    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69" podUID="1e0daaf1-f29c-452c-8cfc-8f4d73137881" containerName="maas-api" containerID="cri-o://59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75" gracePeriod=30
Jul 13 20:02:09.422711 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.422683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:09.641186 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.641130    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:02:09.777852 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.777826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e0daaf1-f29c-452c-8cfc-8f4d73137881-openshift-service-ca\") pod \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") "
Jul 13 20:02:09.777852 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.777862    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz7kn\" (UniqueName: \"kubernetes.io/projected/1e0daaf1-f29c-452c-8cfc-8f4d73137881-kube-api-access-fz7kn\") pod \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") "
Jul 13 20:02:09.778058 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.777883    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e0daaf1-f29c-452c-8cfc-8f4d73137881-maas-api-tls\") pod \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\" (UID: \"1e0daaf1-f29c-452c-8cfc-8f4d73137881\") "
Jul 13 20:02:09.778223 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.778202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e0daaf1-f29c-452c-8cfc-8f4d73137881-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e0daaf1-f29c-452c-8cfc-8f4d73137881" (UID: "1e0daaf1-f29c-452c-8cfc-8f4d73137881"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:02:09.780180 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.780155    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e0daaf1-f29c-452c-8cfc-8f4d73137881-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e0daaf1-f29c-452c-8cfc-8f4d73137881" (UID: "1e0daaf1-f29c-452c-8cfc-8f4d73137881"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:02:09.780284 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.780179    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0daaf1-f29c-452c-8cfc-8f4d73137881-kube-api-access-fz7kn" (OuterVolumeSpecName: "kube-api-access-fz7kn") pod "1e0daaf1-f29c-452c-8cfc-8f4d73137881" (UID: "1e0daaf1-f29c-452c-8cfc-8f4d73137881"). InnerVolumeSpecName "kube-api-access-fz7kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:02:09.879130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.879103    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e0daaf1-f29c-452c-8cfc-8f4d73137881-openshift-service-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:09.879130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.879126    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz7kn\" (UniqueName: \"kubernetes.io/projected/1e0daaf1-f29c-452c-8cfc-8f4d73137881-kube-api-access-fz7kn\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:09.879130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:09.879136    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e0daaf1-f29c-452c-8cfc-8f4d73137881-maas-api-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:10.145267 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.145235    2562 generic.go:356] "Generic (PLEG): container finished" podID="1e0daaf1-f29c-452c-8cfc-8f4d73137881" containerID="59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75" exitCode=0
Jul 13 20:02:10.145436 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.145281    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"
Jul 13 20:02:10.145436 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.145315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69" event={"ID":"1e0daaf1-f29c-452c-8cfc-8f4d73137881","Type":"ContainerDied","Data":"59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75"}
Jul 13 20:02:10.145529 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.145446    2562 scope.go:117] "RemoveContainer" containerID="59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75"
Jul 13 20:02:10.145606 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.145392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69" event={"ID":"1e0daaf1-f29c-452c-8cfc-8f4d73137881","Type":"ContainerDied","Data":"cb0f575cdad486f1ccfde3f6efdd2324bdf6c5fe800b28e66a902051cabde37e"}
Jul 13 20:02:10.153824 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.153810    2562 scope.go:117] "RemoveContainer" containerID="59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75"
Jul 13 20:02:10.154097 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:02:10.154074    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75\": container with ID starting with 59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75 not found: ID does not exist" containerID="59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75"
Jul 13 20:02:10.154194 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.154101    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75"} err="failed to get container status \"59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75\": rpc error: code = NotFound desc = could not find container \"59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75\": container with ID starting with 59738664d6cc8603891a58a1161a4801b7747817c7d7a6a562538c88d2310d75 not found: ID does not exist"
Jul 13 20:02:10.162205 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.162181    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"]
Jul 13 20:02:10.166194 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:10.166171    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7b8c26-5f4d97df5b-4ng69"]
Jul 13 20:02:12.090809 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:12.090771    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e0daaf1-f29c-452c-8cfc-8f4d73137881" path="/var/lib/kubelet/pods/1e0daaf1-f29c-452c-8cfc-8f4d73137881/volumes"
Jul 13 20:02:14.053214 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:14.053179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:19.782623 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:19.782585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:19.851347 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:19.851312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:24.791107 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:24.791078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:25.018636 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.018597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:25.364338 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.364305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:25.664272 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664198    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"]
Jul 13 20:02:25.664529 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664518    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a662bb-c254-4ccf-a33c-2324f713e786" containerName="maas-api"
Jul 13 20:02:25.664594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664557    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a662bb-c254-4ccf-a33c-2324f713e786" containerName="maas-api"
Jul 13 20:02:25.664594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664577    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0daaf1-f29c-452c-8cfc-8f4d73137881" containerName="maas-api"
Jul 13 20:02:25.664594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664582    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0daaf1-f29c-452c-8cfc-8f4d73137881" containerName="maas-api"
Jul 13 20:02:25.664594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664590    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664596    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664604    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664610    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664664    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0daaf1-f29c-452c-8cfc-8f4d73137881" containerName="maas-api"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664672    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a662bb-c254-4ccf-a33c-2324f713e786" containerName="maas-api"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664677    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy"
Jul 13 20:02:25.664715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.664685    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e42a1a-66e6-4552-a0f1-826977228216" containerName="istio-proxy"
Jul 13 20:02:25.667310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.667288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.670211 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.670192    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-519ff3-openshift-default-dockercfg-v5fgc\""
Jul 13 20:02:25.682297 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.682275    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"]
Jul 13 20:02:25.705399 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.705351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:25.711451 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-certs\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5acf42c1-66c5-43f7-9b19-3f3024505453-istiod-ca-cert\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-socket\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-envoy\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-podinfo\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-token\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv774\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-kube-api-access-rv774\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711712    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-data\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.711866 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.711750    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-credential-socket\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.812769 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812734    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-envoy\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.812769 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-podinfo\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-token\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv774\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-kube-api-access-rv774\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812866    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-data\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812917    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-credential-socket\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-certs\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.812975    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5acf42c1-66c5-43f7-9b19-3f3024505453-istiod-ca-cert\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813217 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.813009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-socket\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813596 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.813362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-credential-socket\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813596 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.813441    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-certs\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813596 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.813491    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-data\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.813706 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.813620    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-socket\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.814103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.814081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5acf42c1-66c5-43f7-9b19-3f3024505453-istiod-ca-cert\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.815253 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.815229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-envoy\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.815396 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.815380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-podinfo\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.821400 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.821372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-token\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.821659 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.821639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv774\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-kube-api-access-rv774\") pod \"e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") " pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:25.905736 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.905707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:25.977092 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:25.977033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:26.108050 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.108023    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"]
Jul 13 20:02:26.110006 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:02:26.109981    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5acf42c1_66c5_43f7_9b19_3f3024505453.slice/crio-c43de1900c03c50051875a4ca938f955131a84f4fd7aac0a1e199564636f4ab5 WatchSource:0}: Error finding container c43de1900c03c50051875a4ca938f955131a84f4fd7aac0a1e199564636f4ab5: Status 404 returned error can't find the container with id c43de1900c03c50051875a4ca938f955131a84f4fd7aac0a1e199564636f4ab5
Jul 13 20:02:26.111838 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.111807    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:26.111900 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.111871    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:26.111900 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.111898    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:26.216787 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.216746    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" event={"ID":"5acf42c1-66c5-43f7-9b19-3f3024505453","Type":"ContainerStarted","Data":"f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"}
Jul 13 20:02:26.216954 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.216849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" event={"ID":"5acf42c1-66c5-43f7-9b19-3f3024505453","Type":"ContainerStarted","Data":"c43de1900c03c50051875a4ca938f955131a84f4fd7aac0a1e199564636f4ab5"}
Jul 13 20:02:26.237775 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.237516    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podStartSLOduration=1.237497692 podStartE2EDuration="1.237497692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:02:26.236188381 +0000 UTC m=+2442.651464565" watchObservedRunningTime="2026-07-13 20:02:26.237497692 +0000 UTC m=+2442.652773875"
Jul 13 20:02:26.977617 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:26.977581    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:27.982158 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:27.982128    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:28.170252 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.170215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:28.225905 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.225864    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:28.226945 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.226922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:28.257130 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.257055    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"]
Jul 13 20:02:28.260520 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.260500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.263018 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.262999    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\""
Jul 13 20:02:28.263243 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.263226    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-519ff3-serving-cert\""
Jul 13 20:02:28.269347 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.269300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:28.273274 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.273252    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"]
Jul 13 20:02:28.335323 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.335290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgbt8\" (UniqueName: \"kubernetes.io/projected/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-kube-api-access-vgbt8\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.335470 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.335343    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.335530 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.335492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.352657 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.352631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:28.436192 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.436152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.436382 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.436208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgbt8\" (UniqueName: \"kubernetes.io/projected/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-kube-api-access-vgbt8\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.436382 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.436260    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.436492 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:02:28.436425    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-serving-cert: secret "maas-api-e2e-api-a-519ff3-serving-cert" not found
Jul 13 20:02:28.436492 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:02:28.436480    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls podName:c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4 nodeName:}" failed. No retries permitted until 2026-07-13 20:02:28.936464312 +0000 UTC m=+2445.351740474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls") pod "maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" (UID: "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4") : secret "maas-api-e2e-api-a-519ff3-serving-cert" not found
Jul 13 20:02:28.437064 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.437042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.446062 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.446042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgbt8\" (UniqueName: \"kubernetes.io/projected/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-kube-api-access-vgbt8\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.509488 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.509424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:28.867083 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.867051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:28.941169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.941144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.943706 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.943687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls\") pod \"maas-api-e2e-api-a-519ff3-595cd69f76-msjmm\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:28.969297 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:28.969268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:29.173832 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:29.173737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:29.312072 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:29.312047    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"]
Jul 13 20:02:29.313939 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:02:29.313911    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e02f5a_77e4_4ad7_8946_68a96c8ef3e4.slice/crio-cf7f4dc223a3b2675533ac309ca9bf6ee1ea293d270cbab987975c72ac4dfa37 WatchSource:0}: Error finding container cf7f4dc223a3b2675533ac309ca9bf6ee1ea293d270cbab987975c72ac4dfa37: Status 404 returned error can't find the container with id cf7f4dc223a3b2675533ac309ca9bf6ee1ea293d270cbab987975c72ac4dfa37
Jul 13 20:02:29.315376 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:29.315360    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 20:02:30.237070 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:30.236984    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" event={"ID":"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4","Type":"ContainerStarted","Data":"5c3029db42ab719fb17a4986ee35027c9a9ffbd9c8153c580e10c5383683816c"}
Jul 13 20:02:30.237070 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:30.237033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" event={"ID":"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4","Type":"ContainerStarted","Data":"cf7f4dc223a3b2675533ac309ca9bf6ee1ea293d270cbab987975c72ac4dfa37"}
Jul 13 20:02:30.256374 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:30.256326    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" podStartSLOduration=1.692284844 podStartE2EDuration="2.256310984s" totalImagesPullingTime="564.02614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:02:30.254356646 +0000 UTC m=+2446.669632829" watchObservedRunningTime="2026-07-13 20:02:30.256310984 +0000 UTC m=+2446.671587165"
Jul 13 20:02:31.241039 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:31.241008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:33.230335 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.230301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:33.330789 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.330739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:33.424914 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.424879    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"]
Jul 13 20:02:33.428728 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.428708    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.431144 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.431128    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f221d9-openshift-default-dockercfg-zvg64\""
Jul 13 20:02:33.439455 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.439435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"]
Jul 13 20:02:33.465194 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.465163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:33.478349 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478325    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-certs\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478440 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-credential-socket\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478440 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-data\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478515 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvsvx\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-kube-api-access-gvsvx\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478515 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478500    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-podinfo\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478633 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-socket\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478633 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-envoy\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478633 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-token\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.478633 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.478620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istiod-ca-cert\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.575908 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.575854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:33.580033 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-podinfo\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580101 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-socket\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580101 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-envoy\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580101 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-token\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580388 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istiod-ca-cert\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580388 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-certs\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580388 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-credential-socket\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580388 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-data\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580388 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvsvx\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-kube-api-access-gvsvx\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.580770 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.580676    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-data\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.581106 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.581082    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-socket\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.581203 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.581089    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istiod-ca-cert\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.581203 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.581179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-certs\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.581287 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.581227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-credential-socket\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.582947 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.582926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-podinfo\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.583174 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.583155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-envoy\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.587824 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.587802    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-token\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.587990 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.587974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvsvx\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-kube-api-access-gvsvx\") pod \"e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") " pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.738661 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.738633    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:33.876841 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.876817    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"]
Jul 13 20:02:33.879192 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:02:33.879166    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb6d52f_2de7_411a_bad3_f3b342000ecb.slice/crio-3deb3370c14b11f9bb31e3c05eda461c5de4cd8ea1b55948a1a56846987dbf58 WatchSource:0}: Error finding container 3deb3370c14b11f9bb31e3c05eda461c5de4cd8ea1b55948a1a56846987dbf58: Status 404 returned error can't find the container with id 3deb3370c14b11f9bb31e3c05eda461c5de4cd8ea1b55948a1a56846987dbf58
Jul 13 20:02:33.881839 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.881804    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:33.881924 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.881889    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:33.881924 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:33.881917    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:34.254557 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:34.254447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" event={"ID":"ceb6d52f-2de7-411a-bad3-f3b342000ecb","Type":"ContainerStarted","Data":"f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"}
Jul 13 20:02:34.254557 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:34.254515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" event={"ID":"ceb6d52f-2de7-411a-bad3-f3b342000ecb","Type":"ContainerStarted","Data":"3deb3370c14b11f9bb31e3c05eda461c5de4cd8ea1b55948a1a56846987dbf58"}
Jul 13 20:02:34.278979 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:34.278930    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podStartSLOduration=1.278915896 podStartE2EDuration="1.278915896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:02:34.277190812 +0000 UTC m=+2450.692466996" watchObservedRunningTime="2026-07-13 20:02:34.278915896 +0000 UTC m=+2450.694192076"
Jul 13 20:02:34.739824 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:34.739795    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:34.745190 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:34.745163    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:35.258506 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:35.258480    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:35.259367 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:35.259351    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:36.593244 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:36.593206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:36.690925 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:36.690873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:36.789676 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:36.789636    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:37.257911 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:37.257880    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:38.788253 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:38.788206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:38.889981 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:38.889941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:39.123851 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:39.123783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:39.179777 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:39.179715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:40.722395 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:40.722323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:43.587447 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:43.586737    2562 patch_prober.go:28] interesting pod/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:60804->10.133.0.45:15021: read: connection reset by peer" start-of-body=
Jul 13 20:02:43.587447 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:43.586817    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:60804->10.133.0.45:15021: read: connection reset by peer"
Jul 13 20:02:44.304143 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:44.304110    2562 generic.go:356] "Generic (PLEG): container finished" podID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerID="f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed" exitCode=0
Jul 13 20:02:44.304304 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:44.304183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" event={"ID":"5acf42c1-66c5-43f7-9b19-3f3024505453","Type":"ContainerDied","Data":"f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"}
Jul 13 20:02:44.304599 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:44.304584    2562 scope.go:117] "RemoveContainer" containerID="f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"
Jul 13 20:02:44.305560 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:44.305519    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:44.305667 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:44.305618    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:44.305729 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:44.305690    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:45.310069 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:45.310031    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" event={"ID":"5acf42c1-66c5-43f7-9b19-3f3024505453","Type":"ContainerStarted","Data":"9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75"}
Jul 13 20:02:45.977758 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:45.977709    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:45.977986 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:45.977916    2562 patch_prober.go:28] interesting pod/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 13 20:02:45.978054 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:45.977981    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 13 20:02:46.978563 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:46.977981    2562 patch_prober.go:28] interesting pod/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 13 20:02:46.978563 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:46.978048    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 13 20:02:47.977969 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:47.977920    2562 patch_prober.go:28] interesting pod/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 13 20:02:47.978194 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:47.978005    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 13 20:02:48.977903 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:48.977852    2562 patch_prober.go:28] interesting pod/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 13 20:02:48.978419 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:48.977926    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 13 20:02:49.488981 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:49.488943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:49.588823 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:49.588762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:49.981844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:49.981809    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:49.982233 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:49.982084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:49.982896 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:49.982874    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:02:51.299679 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:51.299632    2562 patch_prober.go:28] interesting pod/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 13 20:02:51.300052 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:51.299691    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 13 20:02:53.348171 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:53.348141    2562 generic.go:356] "Generic (PLEG): container finished" podID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerID="f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00" exitCode=0
Jul 13 20:02:53.348586 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:53.348211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" event={"ID":"ceb6d52f-2de7-411a-bad3-f3b342000ecb","Type":"ContainerDied","Data":"f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"}
Jul 13 20:02:53.348640 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:53.348624    2562 scope.go:117] "RemoveContainer" containerID="f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"
Jul 13 20:02:53.349673 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:53.349650    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:53.349739 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:53.349724    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:53.349777 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:53.349754    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:02:54.359266 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:54.359228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" event={"ID":"ceb6d52f-2de7-411a-bad3-f3b342000ecb","Type":"ContainerStarted","Data":"8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d"}
Jul 13 20:02:54.739574 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:54.739457    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:54.739574 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:54.739515    2562 patch_prober.go:28] interesting pod/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 13 20:02:54.739876 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:54.739589    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 13 20:02:55.739870 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:55.739370    2562 patch_prober.go:28] interesting pod/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 13 20:02:55.739870 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:55.739434    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 13 20:02:56.739715 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:56.739663    2562 patch_prober.go:28] interesting pod/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 13 20:02:56.740495 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:56.740103    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 13 20:02:57.740067 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:57.740027    2562 patch_prober.go:28] interesting pod/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 13 20:02:57.740278 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:57.740093    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 13 20:02:58.475178 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:58.475145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:58.575964 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:58.575926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:58.743350 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:58.743269    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:58.743566 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:58.743531    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:58.744152 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:58.744136    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:02:59.137524 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.137492    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"]
Jul 13 20:02:59.137780 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.137755    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" podUID="c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" containerName="maas-api" containerID="cri-o://5c3029db42ab719fb17a4986ee35027c9a9ffbd9c8153c580e10c5383683816c" gracePeriod=30
Jul 13 20:02:59.160206 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.160183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:02:59.384975 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.384949    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" containerID="5c3029db42ab719fb17a4986ee35027c9a9ffbd9c8153c580e10c5383683816c" exitCode=0
Jul 13 20:02:59.385093 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.385025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" event={"ID":"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4","Type":"ContainerDied","Data":"5c3029db42ab719fb17a4986ee35027c9a9ffbd9c8153c580e10c5383683816c"}
Jul 13 20:02:59.385093 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.385077    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm" event={"ID":"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4","Type":"ContainerDied","Data":"cf7f4dc223a3b2675533ac309ca9bf6ee1ea293d270cbab987975c72ac4dfa37"}
Jul 13 20:02:59.385169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.385101    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf7f4dc223a3b2675533ac309ca9bf6ee1ea293d270cbab987975c72ac4dfa37"
Jul 13 20:02:59.390737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.390676    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:02:59.544208 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.544173    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgbt8\" (UniqueName: \"kubernetes.io/projected/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-kube-api-access-vgbt8\") pod \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") "
Jul 13 20:02:59.544700 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.544216    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls\") pod \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") "
Jul 13 20:02:59.544700 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.544245    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-openshift-service-ca\") pod \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\" (UID: \"c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4\") "
Jul 13 20:02:59.544700 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.544659    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" (UID: "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:02:59.547012 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.546984    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" (UID: "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:02:59.547012 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.546988    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-kube-api-access-vgbt8" (OuterVolumeSpecName: "kube-api-access-vgbt8") pod "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" (UID: "c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4"). InnerVolumeSpecName "kube-api-access-vgbt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:02:59.645418 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.645340    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-openshift-service-ca\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:59.645418 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.645364    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgbt8\" (UniqueName: \"kubernetes.io/projected/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-kube-api-access-vgbt8\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:02:59.645418 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:02:59.645373    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4-maas-api-tls\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:00.389598 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:00.389517    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"
Jul 13 20:03:00.407451 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:00.407419    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"]
Jul 13 20:03:00.411419 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:00.411398    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519ff3-595cd69f76-msjmm"]
Jul 13 20:03:02.090739 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:02.090703    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" path="/var/lib/kubelet/pods/c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4/volumes"
Jul 13 20:03:03.978953 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:03.978913    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:09.773485 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:09.773444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:09.873559 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:09.873504    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:14.652477 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:14.652434    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"]
Jul 13 20:03:14.652979 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:14.652699    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy" containerID="cri-o://9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75" gracePeriod=30
Jul 13 20:03:14.667726 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:14.667697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:14.911637 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:14.911531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:19.910001 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:19.909967    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:03:20.025897 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025814    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5acf42c1-66c5-43f7-9b19-3f3024505453-istiod-ca-cert\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.025897 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-podinfo\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.025897 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025870    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-socket\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025906    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-credential-socket\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025923    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-data\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv774\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-kube-api-access-rv774\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025972    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-certs\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.025998    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-token\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026169 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.026052    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-envoy\") pod \"5acf42c1-66c5-43f7-9b19-3f3024505453\" (UID: \"5acf42c1-66c5-43f7-9b19-3f3024505453\") "
Jul 13 20:03:20.026461 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.026243    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:20.026461 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.026375    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:20.026461 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.026303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5acf42c1-66c5-43f7-9b19-3f3024505453-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:03:20.026662 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.026508    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:20.027056 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.027036    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-data" (OuterVolumeSpecName: "istio-data") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:20.028892 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.028869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-kube-api-access-rv774" (OuterVolumeSpecName: "kube-api-access-rv774") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "kube-api-access-rv774". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:03:20.028972 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.028909    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:20.028972 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.028912    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-token" (OuterVolumeSpecName: "istio-token") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:03:20.028972 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.028908    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5acf42c1-66c5-43f7-9b19-3f3024505453" (UID: "5acf42c1-66c5-43f7-9b19-3f3024505453"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 20:03:20.127136 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127103    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5acf42c1-66c5-43f7-9b19-3f3024505453-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127136 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127132    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127146    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127159    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127174    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127187    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv774\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-kube-api-access-rv774\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127201    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127213    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.127310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.127225    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5acf42c1-66c5-43f7-9b19-3f3024505453-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:20.477970 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.477941    2562 generic.go:356] "Generic (PLEG): container finished" podID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerID="9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75" exitCode=0
Jul 13 20:03:20.478122 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.477988    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"
Jul 13 20:03:20.478122 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.478026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" event={"ID":"5acf42c1-66c5-43f7-9b19-3f3024505453","Type":"ContainerDied","Data":"9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75"}
Jul 13 20:03:20.478122 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.478076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j" event={"ID":"5acf42c1-66c5-43f7-9b19-3f3024505453","Type":"ContainerDied","Data":"c43de1900c03c50051875a4ca938f955131a84f4fd7aac0a1e199564636f4ab5"}
Jul 13 20:03:20.478122 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.478098    2562 scope.go:117] "RemoveContainer" containerID="9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75"
Jul 13 20:03:20.487072 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.487052    2562 scope.go:117] "RemoveContainer" containerID="f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"
Jul 13 20:03:20.495233 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.495209    2562 scope.go:117] "RemoveContainer" containerID="9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75"
Jul 13 20:03:20.495522 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:03:20.495503    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75\": container with ID starting with 9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75 not found: ID does not exist" containerID="9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75"
Jul 13 20:03:20.495682 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.495553    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75"} err="failed to get container status \"9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75\": rpc error: code = NotFound desc = could not find container \"9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75\": container with ID starting with 9ab5b870c21a54446e85d027781543d6f84f0408a68ba5575e0a467fd312ef75 not found: ID does not exist"
Jul 13 20:03:20.495682 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.495578    2562 scope.go:117] "RemoveContainer" containerID="f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"
Jul 13 20:03:20.495856 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:03:20.495839    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed\": container with ID starting with f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed not found: ID does not exist" containerID="f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"
Jul 13 20:03:20.495909 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.495865    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed"} err="failed to get container status \"f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed\": rpc error: code = NotFound desc = could not find container \"f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed\": container with ID starting with f3c68c8f03fd16d63fecc6425cd2791a2b7681a2e13493ae8d86b186ffb4e6ed not found: ID does not exist"
Jul 13 20:03:20.496552 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.496513    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"]
Jul 13 20:03:20.501862 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:20.501838    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-519ff3-openshift-default-788bbc5cff-rsc8j"]
Jul 13 20:03:22.091553 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:22.091495    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" path="/var/lib/kubelet/pods/5acf42c1-66c5-43f7-9b19-3f3024505453/volumes"
Jul 13 20:03:25.511822 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:25.511771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:30.178843 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:30.178807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:35.012940 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:35.012859    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:35.743966 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:35.743907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:35.809711 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:35.809676    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:40.799206 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:40.799166    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"]
Jul 13 20:03:40.799648 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:40.799429    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" containerID="cri-o://8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d" gracePeriod=30
Jul 13 20:03:40.814346 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:40.814312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:41.065113 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.065068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:41.429879 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.429791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:41.721290 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721213    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"]
Jul 13 20:03:41.721620 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721605    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy"
Jul 13 20:03:41.721670 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721621    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy"
Jul 13 20:03:41.721670 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721636    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" containerName="maas-api"
Jul 13 20:03:41.721670 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721643    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" containerName="maas-api"
Jul 13 20:03:41.721670 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721649    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy"
Jul 13 20:03:41.721670 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721654    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy"
Jul 13 20:03:41.721844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721714    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy"
Jul 13 20:03:41.721844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721721    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e02f5a-77e4-4ad7-8946-68a96c8ef3e4" containerName="maas-api"
Jul 13 20:03:41.721844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.721729    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5acf42c1-66c5-43f7-9b19-3f3024505453" containerName="istio-proxy"
Jul 13 20:03:41.726144 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.726124    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.728765 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.728748    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-452240-openshift-default-dockercfg-lt56q\""
Jul 13 20:03:41.736740 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.736719    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"]
Jul 13 20:03:41.749116 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.749092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:41.818554 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-certs\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818574    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-socket\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-credential-socket\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-podinfo\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-token\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-data\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818786    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-envoy\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818928    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgkst\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-kube-api-access-xgkst\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.819110 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.818970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abc79a68-99c8-4d1b-95f7-36a410c463fe-istiod-ca-cert\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.919870 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.919831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-envoy\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.919894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgkst\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-kube-api-access-xgkst\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.919921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abc79a68-99c8-4d1b-95f7-36a410c463fe-istiod-ca-cert\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.919965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-certs\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.919996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-socket\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-credential-socket\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-podinfo\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920378 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-token\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920378 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920139    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-data\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920484 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-certs\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920484 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-socket\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920620 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-data\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920745 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-credential-socket\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.920784 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.920763    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abc79a68-99c8-4d1b-95f7-36a410c463fe-istiod-ca-cert\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.923004 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.922979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-envoy\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.923161 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.923147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-podinfo\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.927662 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.927640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-token\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.927928 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.927911    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgkst\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-kube-api-access-xgkst\") pod \"e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") " pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:41.944656 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:41.944628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:42.037369 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.037268    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:42.202162 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.202051    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"]
Jul 13 20:03:42.204854 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:03:42.204823    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc79a68_99c8_4d1b_95f7_36a410c463fe.slice/crio-3992d30cd8ec31437b7ad03b35ceb547723cd86e5d995bafba70ca05a60130c0 WatchSource:0}: Error finding container 3992d30cd8ec31437b7ad03b35ceb547723cd86e5d995bafba70ca05a60130c0: Status 404 returned error can't find the container with id 3992d30cd8ec31437b7ad03b35ceb547723cd86e5d995bafba70ca05a60130c0
Jul 13 20:03:42.206936 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.206903    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:42.207021 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.206969    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:42.207021 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.207003    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:42.576923 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.576887    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" event={"ID":"abc79a68-99c8-4d1b-95f7-36a410c463fe","Type":"ContainerStarted","Data":"7ddebfde8e15bdbe390921e7ae1c063381e2a72b9f8a8fdf4a9cc03b3f4ce841"}
Jul 13 20:03:42.577141 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.576953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" event={"ID":"abc79a68-99c8-4d1b-95f7-36a410c463fe","Type":"ContainerStarted","Data":"3992d30cd8ec31437b7ad03b35ceb547723cd86e5d995bafba70ca05a60130c0"}
Jul 13 20:03:42.599438 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:42.599362    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podStartSLOduration=1.5993474970000001 podStartE2EDuration="1.599347497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:03:42.597677391 +0000 UTC m=+2519.012953586" watchObservedRunningTime="2026-07-13 20:03:42.599347497 +0000 UTC m=+2519.014623679"
Jul 13 20:03:43.038396 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:43.038301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:43.043129 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:43.043102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:43.587568 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:43.587526    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:43.588705 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:43.588685    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:03:43.745938 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:43.745895    2562 patch_prober.go:28] interesting pod/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 20:03:43.746123 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:43.745963    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 20:03:44.525016 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:44.524924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:44.618337 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:44.618308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:44.716561 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:44.716507    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:44.863283 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:44.863252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:45.281634 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:45.281521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:45.382236 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:45.382192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:46.054188 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.054166    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:03:46.159861 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.159786    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-envoy\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.159861 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.159826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-certs\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.159861 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.159852    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-podinfo\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.159881    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-socket\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.159912    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istiod-ca-cert\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.159986    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-credential-socket\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-data\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvsvx\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-kube-api-access-gvsvx\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160082    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:46.160125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-token\") pod \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\" (UID: \"ceb6d52f-2de7-411a-bad3-f3b342000ecb\") "
Jul 13 20:03:46.160579 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160504    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.160579 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160568    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:46.160919 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.160873    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:46.161173 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.161015    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:03:46.161408 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.161380    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-data" (OuterVolumeSpecName: "istio-data") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:46.162305 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.162274    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-token" (OuterVolumeSpecName: "istio-token") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:03:46.162415 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.162304    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 20:03:46.162415 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.162365    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:03:46.162509 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.162494    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-kube-api-access-gvsvx" (OuterVolumeSpecName: "kube-api-access-gvsvx") pod "ceb6d52f-2de7-411a-bad3-f3b342000ecb" (UID: "ceb6d52f-2de7-411a-bad3-f3b342000ecb"). InnerVolumeSpecName "kube-api-access-gvsvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:03:46.261584 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261554    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261584 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261578    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261584 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261588    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261778 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261597    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261778 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261605    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261778 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261614    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvsvx\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-kube-api-access-gvsvx\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261778 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261623    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.261778 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.261631    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb6d52f-2de7-411a-bad3-f3b342000ecb-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:03:46.602432 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.602401    2562 generic.go:356] "Generic (PLEG): container finished" podID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerID="8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d" exitCode=0
Jul 13 20:03:46.602599 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.602459    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"
Jul 13 20:03:46.602599 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.602486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" event={"ID":"ceb6d52f-2de7-411a-bad3-f3b342000ecb","Type":"ContainerDied","Data":"8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d"}
Jul 13 20:03:46.602599 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.602557    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m" event={"ID":"ceb6d52f-2de7-411a-bad3-f3b342000ecb","Type":"ContainerDied","Data":"3deb3370c14b11f9bb31e3c05eda461c5de4cd8ea1b55948a1a56846987dbf58"}
Jul 13 20:03:46.602599 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.602588    2562 scope.go:117] "RemoveContainer" containerID="8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d"
Jul 13 20:03:46.613819 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.613795    2562 scope.go:117] "RemoveContainer" containerID="f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"
Jul 13 20:03:46.621723 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.621705    2562 scope.go:117] "RemoveContainer" containerID="8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d"
Jul 13 20:03:46.621991 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:03:46.621973    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d\": container with ID starting with 8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d not found: ID does not exist" containerID="8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d"
Jul 13 20:03:46.622034 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.622000    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d"} err="failed to get container status \"8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d\": rpc error: code = NotFound desc = could not find container \"8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d\": container with ID starting with 8fd580f173a3b43148dd1bcc1ed863dc1484097d4501b50f0c2ca36490756b8d not found: ID does not exist"
Jul 13 20:03:46.622034 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.622016    2562 scope.go:117] "RemoveContainer" containerID="f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"
Jul 13 20:03:46.622237 ip-10-0-137-89 kubenswrapper[2562]: E0713 20:03:46.622220    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00\": container with ID starting with f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00 not found: ID does not exist" containerID="f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"
Jul 13 20:03:46.622272 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.622243    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00"} err="failed to get container status \"f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00\": rpc error: code = NotFound desc = could not find container \"f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00\": container with ID starting with f3e4fb6ff82492563e53390298d97c81a20a6511b92cda46460b1402f9107f00 not found: ID does not exist"
Jul 13 20:03:46.626486 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.626466    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"]
Jul 13 20:03:46.630092 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:46.630072    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f221d9-openshift-default-5ddfdc84f7-75z4m"]
Jul 13 20:03:47.994882 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:47.994846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:48.090994 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:48.090959    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" path="/var/lib/kubelet/pods/ceb6d52f-2de7-411a-bad3-f3b342000ecb/volumes"
Jul 13 20:03:49.041720 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.041689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:49.678066 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.678035    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"]
Jul 13 20:03:49.678447 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.678431    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy"
Jul 13 20:03:49.678510 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.678451    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy"
Jul 13 20:03:49.678510 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.678470    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy"
Jul 13 20:03:49.678510 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.678475    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy"
Jul 13 20:03:49.678701 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.678526    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy"
Jul 13 20:03:49.681758 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.681737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.684104 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.684083    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-83eaac-openshift-default-dockercfg-lnbcm\""
Jul 13 20:03:49.692628 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.692606    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"]
Jul 13 20:03:49.782251 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.782218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:49.791446 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-podinfo\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791647 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-token\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791647 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-socket\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791788 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-data\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791788 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-credential-socket\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791788 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-certs\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791942 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lflhw\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-kube-api-access-lflhw\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791942 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-envoy\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.791942 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.791908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/107c07f3-7a69-4a02-bdf2-f7be744b212e-istiod-ca-cert\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.883065 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.883024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:49.892614 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-token\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892752 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-socket\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892752 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-data\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892752 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-credential-socket\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892871 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-certs\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892871 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lflhw\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-kube-api-access-lflhw\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892961 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-envoy\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.892961 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/107c07f3-7a69-4a02-bdf2-f7be744b212e-istiod-ca-cert\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.893064 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.892965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-podinfo\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.893064 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.893017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-socket\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.893064 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.893036    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-credential-socket\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.893241 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.893217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-data\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.893326 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.893302    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-certs\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.893605 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.893584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/107c07f3-7a69-4a02-bdf2-f7be744b212e-istiod-ca-cert\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.895306 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.895287    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-envoy\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.895435 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.895417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-podinfo\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.900727 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.900699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-token\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.900849 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.900832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lflhw\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-kube-api-access-lflhw\") pod \"e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") " pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:49.993168 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:49.993075    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:50.128978 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.128948    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"]
Jul 13 20:03:50.130573 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:03:50.130527    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod107c07f3_7a69_4a02_bdf2_f7be744b212e.slice/crio-7d527db96ded9ae7634bc5ea7ab3cc0505e16f3cb0bee575b5007a3a14c7c752 WatchSource:0}: Error finding container 7d527db96ded9ae7634bc5ea7ab3cc0505e16f3cb0bee575b5007a3a14c7c752: Status 404 returned error can't find the container with id 7d527db96ded9ae7634bc5ea7ab3cc0505e16f3cb0bee575b5007a3a14c7c752
Jul 13 20:03:50.132723 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.132686    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:50.132812 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.132765    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:50.132849 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.132811    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:50.497838 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.497807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:50.624119 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.624079    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" event={"ID":"107c07f3-7a69-4a02-bdf2-f7be744b212e","Type":"ContainerStarted","Data":"e80751330ac304e755140ec20fb1a3667640ce2e4d10d17fd0a0000c1b091763"}
Jul 13 20:03:50.624285 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.624147    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" event={"ID":"107c07f3-7a69-4a02-bdf2-f7be744b212e","Type":"ContainerStarted","Data":"7d527db96ded9ae7634bc5ea7ab3cc0505e16f3cb0bee575b5007a3a14c7c752"}
Jul 13 20:03:50.644261 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.644212    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podStartSLOduration=1.6441948979999998 podStartE2EDuration="1.644194898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:03:50.642971131 +0000 UTC m=+2527.058247315" watchObservedRunningTime="2026-07-13 20:03:50.644194898 +0000 UTC m=+2527.059471081"
Jul 13 20:03:50.993458 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:50.993429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:51.998762 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:51.998734    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:52.634923 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:52.634889    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:52.641957 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:52.641931    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:03:52.795070 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:52.795044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:52.900280 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:52.900204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:52.965528 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:52.965493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:53.407365 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:53.407332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:53.510929 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:53.510898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:53.614475 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:53.614445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:57.219303 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:57.219269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:03:58.589710 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.589680    2562 patch_prober.go:28] interesting pod/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 13 20:03:58.590115 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.589757    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 13 20:03:58.671770 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.671739    2562 generic.go:356] "Generic (PLEG): container finished" podID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerID="7ddebfde8e15bdbe390921e7ae1c063381e2a72b9f8a8fdf4a9cc03b3f4ce841" exitCode=0
Jul 13 20:03:58.671912 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.671802    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" event={"ID":"abc79a68-99c8-4d1b-95f7-36a410c463fe","Type":"ContainerDied","Data":"7ddebfde8e15bdbe390921e7ae1c063381e2a72b9f8a8fdf4a9cc03b3f4ce841"}
Jul 13 20:03:58.672136 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.672123    2562 scope.go:117] "RemoveContainer" containerID="7ddebfde8e15bdbe390921e7ae1c063381e2a72b9f8a8fdf4a9cc03b3f4ce841"
Jul 13 20:03:58.673105 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.673079    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:58.673163 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.673151    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:58.673198 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:58.673180    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:03:59.678104 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:03:59.678072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" event={"ID":"abc79a68-99c8-4d1b-95f7-36a410c463fe","Type":"ContainerStarted","Data":"0018916c702d64385f42958552838c3ac258d35469d3fe63d93f6e5f067aceb3"}
Jul 13 20:04:00.038073 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:00.037961    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:04:00.038257 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:00.038111    2562 patch_prober.go:28] interesting pod/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 13 20:04:00.038257 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:00.038158    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 13 20:04:01.037967 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:01.037930    2562 patch_prober.go:28] interesting pod/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 13 20:04:01.038571 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:01.038520    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 13 20:04:02.038573 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:02.038451    2562 patch_prober.go:28] interesting pod/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 13 20:04:02.039132 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:02.039102    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 13 20:04:03.038374 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:03.038325    2562 patch_prober.go:28] interesting pod/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 13 20:04:03.038613 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:03.038399    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 13 20:04:04.038156 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:04.038108    2562 patch_prober.go:28] interesting pod/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 13 20:04:04.038376 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:04.038178    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 13 20:04:05.042186 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.042081    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:04:05.042698 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.042498    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:04:05.043069 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.043052    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:04:05.708045 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.708016    2562 generic.go:356] "Generic (PLEG): container finished" podID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerID="e80751330ac304e755140ec20fb1a3667640ce2e4d10d17fd0a0000c1b091763" exitCode=0
Jul 13 20:04:05.708224 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.708088    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" event={"ID":"107c07f3-7a69-4a02-bdf2-f7be744b212e","Type":"ContainerDied","Data":"e80751330ac304e755140ec20fb1a3667640ce2e4d10d17fd0a0000c1b091763"}
Jul 13 20:04:05.708618 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.708601    2562 scope.go:117] "RemoveContainer" containerID="e80751330ac304e755140ec20fb1a3667640ce2e4d10d17fd0a0000c1b091763"
Jul 13 20:04:05.709594 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.709565    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:04:05.709667 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.709655    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:04:05.709737 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:05.709721    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 20:04:06.715804 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:06.715766    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" event={"ID":"107c07f3-7a69-4a02-bdf2-f7be744b212e","Type":"ContainerStarted","Data":"cbc51f48de50552136fe6e39236f7670556919ccc3e3889c28f7a816e1f3dae2"}
Jul 13 20:04:06.903557 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:06.903510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:04:06.994587 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:06.993666    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:04:06.994780 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:06.994759    2562 patch_prober.go:28] interesting pod/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 13 20:04:06.994845 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:06.994823    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 13 20:04:07.229395 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:07.229359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:04:07.611763 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:07.611731    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:04:07.634645 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:07.634618    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:04:07.994489 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:07.994399    2562 patch_prober.go:28] interesting pod/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 13 20:04:07.994489 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:07.994457    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 13 20:04:08.993489 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:08.993451    2562 patch_prober.go:28] interesting pod/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 13 20:04:08.993690 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:08.993510    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 13 20:04:09.993638 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:09.993607    2562 patch_prober.go:28] interesting pod/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 13 20:04:09.994089 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:09.994058    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 13 20:04:10.993873 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:10.993834    2562 patch_prober.go:28] interesting pod/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 13 20:04:10.994254 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:10.993901    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 13 20:04:11.997775 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:11.997743    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:04:11.998815 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:11.998798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:04:15.382312 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:15.382276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:04:15.481916 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:15.481884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:04:36.742024 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:36.741994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:04:44.050724 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:04:44.050683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:06:37.869817 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:06:37.869730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:06:37.959428 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:06:37.959389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:06:53.405348 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:06:53.405307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:06:54.331853 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:06:54.331816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:07:12.693563 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:07:12.693505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:07:39.935434 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:07:39.935393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:07:53.178852 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:07:53.178813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:08:27.539249 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:08:27.539212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:08:28.647316 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:08:28.647273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:08:42.503260 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:08:42.503224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:08:44.249146 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:08:44.249112    2562 scope.go:117] "RemoveContainer" containerID="5c3029db42ab719fb17a4986ee35027c9a9ffbd9c8153c580e10c5383683816c"
Jul 13 20:09:08.568842 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:09:08.568787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:09:08.669305 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:09:08.669269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:09:15.734830 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:09:15.734785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:09:16.671254 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:09:16.671221    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:09:16.740847 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:09:16.740804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:09:25.872051 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:09:25.872003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:04.331023 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:04.330983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:07.229015 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:07.228977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:14.331359 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:14.331323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:14.401607 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:14.401571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:25.118145 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:25.118106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:25.219175 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:25.219141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:35.248202 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:35.248166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:39.657449 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:39.657408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:46.125310 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:46.125270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:46.210108 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:46.210076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:50.346273 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:50.346231    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"]
Jul 13 20:10:50.346641 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:50.346490    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy" containerID="cri-o://0018916c702d64385f42958552838c3ac258d35469d3fe63d93f6e5f067aceb3" gracePeriod=30
Jul 13 20:10:50.386399 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:50.386359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:50.598769 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:50.598666    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:51.429418 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:51.429383    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:51.527221 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:51.527191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:10:55.564928 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.564900    2562 generic.go:356] "Generic (PLEG): container finished" podID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerID="0018916c702d64385f42958552838c3ac258d35469d3fe63d93f6e5f067aceb3" exitCode=0
Jul 13 20:10:55.565266 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.564948    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" event={"ID":"abc79a68-99c8-4d1b-95f7-36a410c463fe","Type":"ContainerDied","Data":"0018916c702d64385f42958552838c3ac258d35469d3fe63d93f6e5f067aceb3"}
Jul 13 20:10:55.565266 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.565036    2562 scope.go:117] "RemoveContainer" containerID="7ddebfde8e15bdbe390921e7ae1c063381e2a72b9f8a8fdf4a9cc03b3f4ce841"
Jul 13 20:10:55.594075 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.594049    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:10:55.610903 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.610857    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-socket\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.610903 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.610886    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-credential-socket\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.610904    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-token\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.610928    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-data\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.610972    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abc79a68-99c8-4d1b-95f7-36a410c463fe-istiod-ca-cert\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.610997    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgkst\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-kube-api-access-xgkst\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611036    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-certs\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611059    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-envoy\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611103 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611089    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-podinfo\") pod \"abc79a68-99c8-4d1b-95f7-36a410c463fe\" (UID: \"abc79a68-99c8-4d1b-95f7-36a410c463fe\") "
Jul 13 20:10:55.611458 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611152    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:55.611458 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611187    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:55.611458 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:55.611458 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611397    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.611458 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611433    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.611749 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611697    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abc79a68-99c8-4d1b-95f7-36a410c463fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:10:55.611904 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.611875    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:55.613230 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.613209    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:10:55.613886 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.613854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 20:10:55.613886 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.613854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:55.614038 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.613885    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-kube-api-access-xgkst" (OuterVolumeSpecName: "kube-api-access-xgkst") pod "abc79a68-99c8-4d1b-95f7-36a410c463fe" (UID: "abc79a68-99c8-4d1b-95f7-36a410c463fe"). InnerVolumeSpecName "kube-api-access-xgkst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:10:55.712229 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712199    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.712229 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712226    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.712229 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712235    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abc79a68-99c8-4d1b-95f7-36a410c463fe-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.712420 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712244    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgkst\" (UniqueName: \"kubernetes.io/projected/abc79a68-99c8-4d1b-95f7-36a410c463fe-kube-api-access-xgkst\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.712420 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712253    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.712420 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712261    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:55.712420 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:55.712269    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abc79a68-99c8-4d1b-95f7-36a410c463fe-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:10:56.570020 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:56.569987    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc" event={"ID":"abc79a68-99c8-4d1b-95f7-36a410c463fe","Type":"ContainerDied","Data":"3992d30cd8ec31437b7ad03b35ceb547723cd86e5d995bafba70ca05a60130c0"}
Jul 13 20:10:56.570020 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:56.570009    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"
Jul 13 20:10:56.570465 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:56.570054    2562 scope.go:117] "RemoveContainer" containerID="0018916c702d64385f42958552838c3ac258d35469d3fe63d93f6e5f067aceb3"
Jul 13 20:10:56.588039 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:56.588017    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"]
Jul 13 20:10:56.593729 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:56.593711    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-452240-openshift-default-dd4778dff-mgtbc"]
Jul 13 20:10:58.091520 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:10:58.091488    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" path="/var/lib/kubelet/pods/abc79a68-99c8-4d1b-95f7-36a410c463fe/volumes"
Jul 13 20:11:01.388226 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:01.388175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:05.803115 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:05.803027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:12.141909 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:12.141871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:12.238451 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:12.238419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:16.470597 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:16.470565    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"]
Jul 13 20:11:16.471022 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:16.470786    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy" containerID="cri-o://cbc51f48de50552136fe6e39236f7670556919ccc3e3889c28f7a816e1f3dae2" gracePeriod=30
Jul 13 20:11:16.500465 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:16.500441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:16.718429 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:16.718396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:21.687836 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.687803    2562 generic.go:356] "Generic (PLEG): container finished" podID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerID="cbc51f48de50552136fe6e39236f7670556919ccc3e3889c28f7a816e1f3dae2" exitCode=0
Jul 13 20:11:21.688251 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.687849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" event={"ID":"107c07f3-7a69-4a02-bdf2-f7be744b212e","Type":"ContainerDied","Data":"cbc51f48de50552136fe6e39236f7670556919ccc3e3889c28f7a816e1f3dae2"}
Jul 13 20:11:21.688251 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.687927    2562 scope.go:117] "RemoveContainer" containerID="e80751330ac304e755140ec20fb1a3667640ce2e4d10d17fd0a0000c1b091763"
Jul 13 20:11:21.720951 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.720929    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:11:21.823288 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823257    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lflhw\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-kube-api-access-lflhw\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823459 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823322    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-podinfo\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823459 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823348    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-token\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823459 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823370    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-credential-socket\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823459 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-socket\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823459 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-data\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823459 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823444    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-certs\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823788 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/107c07f3-7a69-4a02-bdf2-f7be744b212e-istiod-ca-cert\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823788 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823561    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-envoy\") pod \"107c07f3-7a69-4a02-bdf2-f7be744b212e\" (UID: \"107c07f3-7a69-4a02-bdf2-f7be744b212e\") "
Jul 13 20:11:21.823889 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823816    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:11:21.823889 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823831    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:11:21.823994 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823936    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.823994 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.823954    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-workload-certs\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.824182 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.824162    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/107c07f3-7a69-4a02-bdf2-f7be744b212e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:11:21.824329 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.824303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:11:21.824586 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.824525    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-data" (OuterVolumeSpecName: "istio-data") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:11:21.825599 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.825569    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-token" (OuterVolumeSpecName: "istio-token") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:11:21.825894 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.825878    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:11:21.826200 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.826177    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-kube-api-access-lflhw" (OuterVolumeSpecName: "kube-api-access-lflhw") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "kube-api-access-lflhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:11:21.826200 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.826183    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "107c07f3-7a69-4a02-bdf2-f7be744b212e" (UID: "107c07f3-7a69-4a02-bdf2-f7be744b212e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 20:11:21.925014 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.924977    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lflhw\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-kube-api-access-lflhw\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.925014 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.925012    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-podinfo\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.925014 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.925021    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-token\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.925203 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.925030    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-credential-socket\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.925203 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.925038    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-data\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.925203 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.925046    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/107c07f3-7a69-4a02-bdf2-f7be744b212e-istiod-ca-cert\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:21.925203 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:21.925052    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/107c07f3-7a69-4a02-bdf2-f7be744b212e-istio-envoy\") on node \"ip-10-0-137-89.ec2.internal\" DevicePath \"\""
Jul 13 20:11:22.104231 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.104204    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xn9bx_1845848f-2b33-4144-9187-356b9905ec22/manager/0.log"
Jul 13 20:11:22.347878 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.347850    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d8fb89cc8-tdbfm_f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e/manager/0.log"
Jul 13 20:11:22.463164 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.463081    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6ljsh_53826db9-0478-4e1f-9823-a45808e81e83/server/0.log"
Jul 13 20:11:22.694404 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.694373    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"
Jul 13 20:11:22.694404 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.694394    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj" event={"ID":"107c07f3-7a69-4a02-bdf2-f7be744b212e","Type":"ContainerDied","Data":"7d527db96ded9ae7634bc5ea7ab3cc0505e16f3cb0bee575b5007a3a14c7c752"}
Jul 13 20:11:22.694961 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.694461    2562 scope.go:117] "RemoveContainer" containerID="cbc51f48de50552136fe6e39236f7670556919ccc3e3889c28f7a816e1f3dae2"
Jul 13 20:11:22.715109 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.715060    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"]
Jul 13 20:11:22.720675 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.720646    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-83eaac-openshift-default-5f4c7c4555-g5bbj"]
Jul 13 20:11:22.881865 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.881842    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-fgl2c_067fd0f8-6a43-420c-a7db-09b881518dfa/copy-manifests/0.log"
Jul 13 20:11:22.888627 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:22.888604    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-fgl2c_067fd0f8-6a43-420c-a7db-09b881518dfa/manager/0.log"
Jul 13 20:11:23.351248 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:23.351219    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8bd4df569-2r472_4796d3b0-9beb-4383-b6f3-3dae3e3a9015/maas-api/0.log"
Jul 13 20:11:24.090494 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:24.090465    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" path="/var/lib/kubelet/pods/107c07f3-7a69-4a02-bdf2-f7be744b212e/volumes"
Jul 13 20:11:25.031517 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:25.031478    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-286pl_a2052937-bcba-4a1a-936d-bd8bb3301074/postgres/0.log"
Jul 13 20:11:26.845052 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:26.845011    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ftkvv_30c9db42-b3ec-473e-bbe8-9819a02bd534/registry-server/0.log"
Jul 13 20:11:27.212965 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:27.212893    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5rp8m_352a0ab9-f732-40ad-b511-b6c086aecf91/manager/0.log"
Jul 13 20:11:27.682983 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:27.682945    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w2c2q_ac23756b-d61c-4d52-aaa8-42abeb01e095/discovery/0.log"
Jul 13 20:11:27.894157 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:27.894121    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-mxm6s_c3dea633-36df-46bf-a96c-55e0a58f7567/kube-auth-proxy/0.log"
Jul 13 20:11:28.144788 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:28.144734    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mp6wh_b4419eee-d4cb-42b0-8a10-1b90b63ccf3a/payload-pre-processing/0.log"
Jul 13 20:11:28.283047 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:28.283016    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-x79dq_bb32bdc3-2727-4d0c-b1d8-be866aa42947/payload-processing/0.log"
Jul 13 20:11:28.742923 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:28.742881    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-fxcqb_200d9b7a-7aa2-45fe-8446-ed5ced9da545/main/0.log"
Jul 13 20:11:28.863586 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:28.863528    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-n726q_43b5d145-3928-4a94-8985-488ecb102253/main/0.log"
Jul 13 20:11:29.211351 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:29.211325    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nbpnt_f5aa1169-7397-4d14-835f-1daf8ff417d5/main/0.log"
Jul 13 20:11:33.022111 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:33.022069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:33.125122 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:33.125091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:36.177708 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:36.177679    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-276kq_e22396dc-eaa1-48db-b917-bfea50c1c49c/global-pull-secret-syncer/0.log"
Jul 13 20:11:36.335494 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:36.335461    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log"
Jul 13 20:11:36.477648 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:36.477565    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-89.ec2.internal_337d027ddbda42a4cd3b3abf536a25de/haproxy/0.log"
Jul 13 20:11:38.724654 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:38.724617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:38.832975 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:38.832947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rq9wv_da0e4894-383d-462d-b290-8d05c93f3ad6/konnectivity-agent/0.log" limit=1024
Jul 13 20:11:42.954407 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:42.954376    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ftkvv_30c9db42-b3ec-473e-bbe8-9819a02bd534/registry-server/0.log"
Jul 13 20:11:43.291231 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:43.291154    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5rp8m_352a0ab9-f732-40ad-b511-b6c086aecf91/manager/0.log"
Jul 13 20:11:45.453477 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:45.453449    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g2rmr_c8efa588-1365-42d2-b35c-bb23fab7aa0b/node-exporter/0.log"
Jul 13 20:11:45.488351 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:45.488328    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g2rmr_c8efa588-1365-42d2-b35c-bb23fab7aa0b/kube-rbac-proxy/0.log"
Jul 13 20:11:45.512012 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:45.511992    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g2rmr_c8efa588-1365-42d2-b35c-bb23fab7aa0b/init-textfile/0.log"
Jul 13 20:11:47.684009 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.683973    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"]
Jul 13 20:11:47.684456 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684423    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy"
Jul 13 20:11:47.684456 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684440    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy"
Jul 13 20:11:47.684610 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684459    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy"
Jul 13 20:11:47.684610 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684468    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy"
Jul 13 20:11:47.684610 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684481    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy"
Jul 13 20:11:47.684610 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684489    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy"
Jul 13 20:11:47.684610 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684607    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb6d52f-2de7-411a-bad3-f3b342000ecb" containerName="istio-proxy"
Jul 13 20:11:47.684844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684621    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy"
Jul 13 20:11:47.684844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684630    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="107c07f3-7a69-4a02-bdf2-f7be744b212e" containerName="istio-proxy"
Jul 13 20:11:47.684844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684640    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy"
Jul 13 20:11:47.684844 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.684648    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="abc79a68-99c8-4d1b-95f7-36a410c463fe" containerName="istio-proxy"
Jul 13 20:11:47.688906 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.688883    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.692415 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.692393    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dl6jf\"/\"default-dockercfg-pfg74\""
Jul 13 20:11:47.692520 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.692400    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dl6jf\"/\"openshift-service-ca.crt\""
Jul 13 20:11:47.693301 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.693278    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dl6jf\"/\"kube-root-ca.crt\""
Jul 13 20:11:47.696690 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.696668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"]
Jul 13 20:11:47.832344 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.832300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-proc\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.832344 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.832345    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbn9m\" (UniqueName: \"kubernetes.io/projected/acae56f8-04d5-484a-9e6e-cffea0f67ffd-kube-api-access-sbn9m\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.832630 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.832458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-podres\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.832630 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.832514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-sys\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.832630 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.832575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-lib-modules\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.933901 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.933857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-podres\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934099 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.933916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-sys\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934099 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.933949    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-lib-modules\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934099 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.933967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-proc\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934099 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.933984    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbn9m\" (UniqueName: \"kubernetes.io/projected/acae56f8-04d5-484a-9e6e-cffea0f67ffd-kube-api-access-sbn9m\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934099 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.934033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-podres\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934099 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.934087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-sys\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934379 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.934087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-proc\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.934379 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.934115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acae56f8-04d5-484a-9e6e-cffea0f67ffd-lib-modules\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.943219 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.943194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbn9m\" (UniqueName: \"kubernetes.io/projected/acae56f8-04d5-484a-9e6e-cffea0f67ffd-kube-api-access-sbn9m\") pod \"perf-node-gather-daemonset-dplbk\" (UID: \"acae56f8-04d5-484a-9e6e-cffea0f67ffd\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:47.998832 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:47.998808    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:48.137084 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:48.137061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"]
Jul 13 20:11:48.139549 ip-10-0-137-89 kubenswrapper[2562]: W0713 20:11:48.139510    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podacae56f8_04d5_484a_9e6e_cffea0f67ffd.slice/crio-7a8c6f935812f67c2ab925d1f77e5c17a17884055e6d279b72de8a6895e9b2dd WatchSource:0}: Error finding container 7a8c6f935812f67c2ab925d1f77e5c17a17884055e6d279b72de8a6895e9b2dd: Status 404 returned error can't find the container with id 7a8c6f935812f67c2ab925d1f77e5c17a17884055e6d279b72de8a6895e9b2dd
Jul 13 20:11:48.806906 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:48.806872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk" event={"ID":"acae56f8-04d5-484a-9e6e-cffea0f67ffd","Type":"ContainerStarted","Data":"8c36e8c884bcd463db50e6db9245085331d31795a1bbb9d2b16fa564180e74f8"}
Jul 13 20:11:48.807305 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:48.806917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk" event={"ID":"acae56f8-04d5-484a-9e6e-cffea0f67ffd","Type":"ContainerStarted","Data":"7a8c6f935812f67c2ab925d1f77e5c17a17884055e6d279b72de8a6895e9b2dd"}
Jul 13 20:11:48.807305 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:48.806983    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:48.830562 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:48.830486    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk" podStartSLOduration=1.830473155 podStartE2EDuration="1.830473155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:48.828569096 +0000 UTC m=+3005.243845295" watchObservedRunningTime="2026-07-13 20:11:48.830473155 +0000 UTC m=+3005.245749370"
Jul 13 20:11:50.515454 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:50.515424    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-llqzj_ab74d8e7-399e-431b-8fbe-d57c1c8a9a59/dns/0.log"
Jul 13 20:11:50.540891 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:50.540870    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-llqzj_ab74d8e7-399e-431b-8fbe-d57c1c8a9a59/kube-rbac-proxy/0.log"
Jul 13 20:11:50.654815 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:50.654790    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6cg2s_94e22fbe-3c4c-4c65-946a-a6a2be93b2b7/dns-node-resolver/0.log"
Jul 13 20:11:51.423030 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:51.422997    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n5b2j_8099d8bb-1c5a-4a53-9bb4-a0b30e8e9071/node-ca/0.log"
Jul 13 20:11:52.730503 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:52.730472    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w2c2q_ac23756b-d61c-4d52-aaa8-42abeb01e095/discovery/0.log"
Jul 13 20:11:52.789920 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:52.789893    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-mxm6s_c3dea633-36df-46bf-a96c-55e0a58f7567/kube-auth-proxy/0.log"
Jul 13 20:11:52.902379 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:52.902346    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mp6wh_b4419eee-d4cb-42b0-8a10-1b90b63ccf3a/payload-pre-processing/0.log"
Jul 13 20:11:52.943615 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:52.943583    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-x79dq_bb32bdc3-2727-4d0c-b1d8-be866aa42947/payload-processing/0.log"
Jul 13 20:11:53.801785 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:53.801752    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l5gft_e4ea392e-7d04-4c8a-bc09-ba81c768126b/serve-healthcheck-canary/0.log"
Jul 13 20:11:54.375784 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:54.375745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ckn56_7b1d8e47-12e4-4dad-b07d-6dadfb2ba146/kube-rbac-proxy/0.log"
Jul 13 20:11:54.398661 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:54.398640    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ckn56_7b1d8e47-12e4-4dad-b07d-6dadfb2ba146/exporter/0.log"
Jul 13 20:11:54.430272 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:54.430242    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ckn56_7b1d8e47-12e4-4dad-b07d-6dadfb2ba146/extractor/0.log"
Jul 13 20:11:54.820997 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:54.820974    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-dplbk"
Jul 13 20:11:56.984950 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:56.984920    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xn9bx_1845848f-2b33-4144-9187-356b9905ec22/manager/0.log"
Jul 13 20:11:57.318045 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.318015    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d8fb89cc8-tdbfm_f266b68e-5fa7-48f7-a113-ea6ddb7f8f6e/manager/0.log"
Jul 13 20:11:57.343125 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.343098    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6ljsh_53826db9-0478-4e1f-9823-a45808e81e83/server/0.log"
Jul 13 20:11:57.556649 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.556625    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w2c2q_ac23756b-d61c-4d52-aaa8-42abeb01e095/discovery/0.log"
Jul 13 20:11:57.606848 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.606814    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-mxm6s_c3dea633-36df-46bf-a96c-55e0a58f7567/kube-auth-proxy/0.log"
Jul 13 20:11:57.643484 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.643455    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-fgl2c_067fd0f8-6a43-420c-a7db-09b881518dfa/manager/0.log"
Jul 13 20:11:57.676294 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.676265    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-fgl2c_067fd0f8-6a43-420c-a7db-09b881518dfa/copy-manifests/0.log"
Jul 13 20:11:57.692642 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.692619    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mp6wh_b4419eee-d4cb-42b0-8a10-1b90b63ccf3a/payload-pre-processing/0.log"
Jul 13 20:11:57.722048 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:57.722022    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-x79dq_bb32bdc3-2727-4d0c-b1d8-be866aa42947/payload-processing/0.log"
Jul 13 20:11:59.215330 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:59.215290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w2c2q_ac23756b-d61c-4d52-aaa8-42abeb01e095/discovery/0.log"
Jul 13 20:11:59.249019 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:59.248943    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-mxm6s_c3dea633-36df-46bf-a96c-55e0a58f7567/kube-auth-proxy/0.log"
Jul 13 20:11:59.333695 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:59.333664    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mp6wh_b4419eee-d4cb-42b0-8a10-1b90b63ccf3a/payload-pre-processing/0.log"
Jul 13 20:11:59.365942 ip-10-0-137-89 kubenswrapper[2562]: I0713 20:11:59.365911    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-x79dq_bb32bdc3-2727-4d0c-b1d8-be866aa42947/payload-processing/0.log"