Jul 23 13:17:49.513813 ip-10-0-136-232 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 13:17:49.892089 ip-10-0-136-232 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 13:17:49.892089 ip-10-0-136-232 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 13:17:49.892089 ip-10-0-136-232 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 13:17:49.892089 ip-10-0-136-232 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 13:17:49.892089 ip-10-0-136-232 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 13:17:49.892904 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.892794    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 13:17:49.898229 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898211    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 13:17:49.898229 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898227    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 13:17:49.898229 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898230    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 13:17:49.898229 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898234    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898237    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898240    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898243    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898245    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898248    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898252    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898255    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898257    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898260    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898263    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898266    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898269    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898272    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898274    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898277    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898279    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898281    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898284    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898287    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 13:17:49.898377 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898290    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898292    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898295    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898297    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898306    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898308    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898311    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898313    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898316    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898318    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898321    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898323    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898326    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898329    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898332    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898334    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898338    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898342    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898345    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 13:17:49.898877 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898348    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898351    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898354    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898356    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898359    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898361    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898364    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898366    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898369    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898371    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898375    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898378    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898381    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898384    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898387    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898390    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898393    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898395    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898398    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 13:17:49.899365 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898400    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898403    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898407    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898410    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898412    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898415    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898417    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898420    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898423    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898426    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898431    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898433    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898436    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898438    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898441    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898443    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898446    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898448    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898451    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898453    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 13:17:49.899849 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898456    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898459    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898461    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898464    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898466    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898468    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898471    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898473    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898476    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898478    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898481    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898485    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898488    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898492    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898495    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898498    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898502    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.898505    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899204    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 13:17:49.900329 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899210    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899215    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899218    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899221    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899225    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899230    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899233    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899237    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899240    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899243    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899245    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899249    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899251    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899254    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899257    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899259    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899262    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899265    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899267    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 13:17:49.900795 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899270    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899272    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899275    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899277    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899280    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899283    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899285    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899288    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899290    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899293    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899295    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899298    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899301    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899303    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899306    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899308    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899311    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899313    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899316    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899318    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 13:17:49.901264 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899321    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899323    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899325    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899328    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899330    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899333    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899335    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899338    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899340    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899343    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899345    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899347    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899350    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899352    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899356    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899358    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899361    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899363    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899366    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899369    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 13:17:49.901778 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899371    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899374    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899377    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899380    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899383    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899386    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899388    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899391    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899393    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899396    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899399    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899401    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899404    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899406    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899409    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899411    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899414    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899417    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899419    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899422    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 13:17:49.902270 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899424    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899427    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899429    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899432    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899434    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899436    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899439    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899443    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899445    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899448    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899450    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899453    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899456    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899467    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899470    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899473    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899476    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899479    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899482    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899854    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 13:17:49.902776 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899859    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899861    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899865    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899868    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899871    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899874    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899877    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899879    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899884    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899887    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899890    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899893    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899895    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899898    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899901    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899903    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899906    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899908    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899911    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899913    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 13:17:49.903267 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899915    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899918    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899920    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899923    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899926    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899928    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899932    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899936    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899939    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899942    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899945    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899948    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899950    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899953    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899956    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899959    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899961    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899964    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899966    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 13:17:49.903767 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899968    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899972    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899975    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899979    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899981    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899984    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899986    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899989    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899991    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899994    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899996    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.899998    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900001    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900003    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900006    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900010    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900014    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900017    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900020    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 13:17:49.904223 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900022    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900025    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900027    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900030    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900033    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900036    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900038    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900040    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900043    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900045    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900047    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900050    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900053    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900055    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900058    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900061    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900063    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900066    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900068    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900071    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 13:17:49.904724 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900073    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900075    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900079    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900081    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900083    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900086    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900088    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900091    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900093    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900096    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900098    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900101    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900103    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900106    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900108    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900110    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900113    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900116    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900118    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900121    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 13:17:49.905208 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900190    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900196    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900203    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900207    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900211    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900214    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900222    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900226    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900230    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900233    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900236    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900240    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900242    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900245    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900248    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900251    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900254    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900256    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900265    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900268    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900271    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900274    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900277    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900281    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 13:17:49.905700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900283    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900287    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900290    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900293    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900296    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900299    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900302    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900305    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900309    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900312    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900315    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900318    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900321    2559 flags.go:64] FLAG: --enable-server="true"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900323    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900329    2559 flags.go:64] FLAG: --event-burst="100"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900332    2559 flags.go:64] FLAG: --event-qps="50"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900335    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900338    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900341    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900349    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900352    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900355    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900357    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900360    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900363    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 13:17:49.906305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900365    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900368    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900371    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900374    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900376    2559 flags.go:64] FLAG: --feature-gates=""
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900380    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900383    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900386    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900389    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900392    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900395    2559 flags.go:64] FLAG: --help="false"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900398    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-136-232.ec2.internal"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900401    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900404    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900407    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900410    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900413    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900416    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900418    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900421    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900424    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900427    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900430    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900435    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 23 13:17:49.906943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900437    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900440    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900443    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900446    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900449    2559 flags.go:64] FLAG: --lock-file=""
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900451    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900454    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900457    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900462    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900465    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900468    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900470    2559 flags.go:64] FLAG: --logging-format="text"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900473    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900476    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900479    2559 flags.go:64] FLAG: --manifest-url=""
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900481    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900485    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900488    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900492    2559 flags.go:64] FLAG: --max-pods="110"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900495    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900497    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900500    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900503    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900506    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900509    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 13:17:49.907544 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900512    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900518    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900521    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900524    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900527    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900547    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900556    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900563    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900567    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900570    2559 flags.go:64] FLAG: --port="10250"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900573    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900576    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05c745eda69c984e2"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900579    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900582    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900585    2559 flags.go:64] FLAG: --register-node="true"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900587    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900591    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900594    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900596    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900599    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900603    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900606    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900608    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900611    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900614    2559 flags.go:64] FLAG: --runonce="false"
Jul 23 13:17:49.908175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900617    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900620    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900623    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900626    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900628    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900631    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900635    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900638    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900641    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900643    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900646    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900649    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900652    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900654    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900659    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900664    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900667    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900669    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900674    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900677    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900679    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900682    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900685    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900688    2559 flags.go:64] FLAG: --v="2"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900692    2559 flags.go:64] FLAG: --version="false"
Jul 23 13:17:49.909054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900696    2559 flags.go:64] FLAG: --vmodule=""
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900699    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.900702    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900832    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900837    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900840    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900843    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900845    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900848    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900851    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900853    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900856    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900858    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900861    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900864    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900866    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900869    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900872    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900874    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900877    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900880    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 13:17:49.909930 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900883    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900886    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900888    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900891    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900894    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900897    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900899    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900902    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900904    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900907    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900909    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900912    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900914    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900917    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900919    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900922    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900925    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900927    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900930    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900932    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 13:17:49.910478 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900935    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900938    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900942    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900944    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900946    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900949    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900954    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900957    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900959    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900961    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900964    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900966    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900970    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900974    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900977    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900980    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900983    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900986    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900988    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 13:17:49.910998 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900991    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900994    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.900997    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901001    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901004    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901007    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901009    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901012    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901016    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901019    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901022    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901024    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901027    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901030    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901032    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901036    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901038    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901041    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901044    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 13:17:49.911466 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901047    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901050    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901052    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901055    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901057    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901059    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901064    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901067    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901069    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901071    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901074    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901077    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901079    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901081    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901084    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901086    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901089    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901091    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901093    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901096    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 13:17:49.911949 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901098    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901101    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.901103    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.901728    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.908360    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.908375    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.908388    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.908394    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908771    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908785    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908790    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908796    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908800    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908805    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908809    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 13:17:49.912441 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908814    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908818    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908830    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908835    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908839    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908844    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908849    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908853    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908857    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908861    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908868    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908875    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908879    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908884    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908889    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908898    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908902    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908907    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908911    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 13:17:49.912850 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908915    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908919    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908924    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908938    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908946    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908951    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908956    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908961    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908970    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908974    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908981    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908985    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908990    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908994    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.908999    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909003    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909007    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909011    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 13:17:49.913308 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909016    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909020    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909029    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909033    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909037    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909042    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909046    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909051    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909055    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909060    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909064    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909069    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909074    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909078    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909083    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909092    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909097    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909102    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909107    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909113    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 13:17:49.913887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909118    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909122    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909129    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909136    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909141    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909146    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909151    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909162    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909166    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909170    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909175    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909179    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909184    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909188    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909192    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909196    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909201    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909205    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909209    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909218    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 13:17:49.914375 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909223    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909227    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909232    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909236    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909241    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909245    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909249    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909253    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909257    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909263    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909267    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909272    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909281    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909285    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909290    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 13:17:49.914894 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.909298    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909854    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909868    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909872    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909875    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909878    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909880    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909884    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909887    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909889    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909892    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909895    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909898    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909900    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909903    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909906    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909908    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909911    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909914    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 13:17:49.915282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909917    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909919    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909922    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909924    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909927    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909929    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909932    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909935    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909937    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909940    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909944    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909947    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909949    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909952    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909954    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909958    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909960    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909963    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909966    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909968    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 13:17:49.915771 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909971    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909974    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909976    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909979    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909981    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909984    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909986    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909989    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909991    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909994    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909996    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.909998    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910002    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910006    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910010    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910013    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910015    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910018    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910021    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 13:17:49.916282 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910024    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910027    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910029    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910032    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910035    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910037    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910040    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910042    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910045    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910048    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910051    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910053    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910056    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910058    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910061    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910063    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910066    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910069    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910071    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910074    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 13:17:49.916774 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910076    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910079    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910081    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910084    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910086    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910088    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910090    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910093    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910095    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910098    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910100    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910103    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910107    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910110    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910112    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910116    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910119    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910122    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910125    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910128    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 13:17:49.917259 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910131    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:49.910134    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.910138    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.910250    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.912277    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.913499    2559 server.go:1019] "Starting client certificate rotation"
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.913580    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.913615    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 13:17:49.917763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.916037    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 13:17:49.920090 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.920073    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 13:17:49.931560 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.931545    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 23 13:17:49.933684 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.933671    2559 log.go:25] "Validated CRI v1 image API"
Jul 23 13:17:49.934494 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.934480    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 13:17:49.938656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.938638    2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 856eea89-3cc5-4272-8b4d-e17c4beee8a2:/dev/nvme0n1p3 a7588fec-1f0f-430b-a89a-72df9115620b:/dev/nvme0n1p4]
Jul 23 13:17:49.938710 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.938656    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 13:17:49.943335 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.943203    2559 manager.go:216] Machine: {Timestamp:2026-07-23 13:17:49.942735065 +0000 UTC m=+0.318685834 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3052776 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec286a114c3fb62b60bf05331dd8fefa SystemUUID:ec286a11-4c3f-b62b-60bf-05331dd8fefa BootID:ee2d2413-303e-4994-aeab-5c7798ac227b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7c:c1:d3:f8:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7c:c1:d3:f8:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:af:63:2a:19:73 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 13:17:49.943335 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.943336    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 13:17:49.943431 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.943410    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 13:17:49.944651 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.944634    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 13:17:49.946403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.946381    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 13:17:49.946582 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.946407    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-232.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 13:17:49.947249 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.947239    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 13:17:49.947279 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.947252    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 13:17:49.947279 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.947260    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 13:17:49.947915 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.947905    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 13:17:49.949053 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.949044    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 13:17:49.952049 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952037    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 13:17:49.952174 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952164    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 13:17:49.952229 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952221    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 13:17:49.952262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952240    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 13:17:49.952262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952250    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 13:17:49.952262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952259    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 23 13:17:49.952342 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952268    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 13:17:49.952815 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952798    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 13:17:49.952864 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.952832    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 13:17:49.956587 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.956564    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 13:17:49.958406 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.958394    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 13:17:49.958469 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.958423    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 13:17:49.959469 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959453    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 13:17:49.959512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959475    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 13:17:49.959512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959481    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 13:17:49.959512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959487    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 13:17:49.959512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959498    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 13:17:49.959512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959507    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 13:17:49.959657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959517    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 13:17:49.959657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959544    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 13:17:49.959657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959556    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 13:17:49.959657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959564    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 13:17:49.959657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959576    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 13:17:49.959657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.959584    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 13:17:49.960291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.960282    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 13:17:49.960359 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.960292    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 13:17:49.963630 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.963613    2559 server.go:1268] "Started kubelet"
Jul 23 13:17:49.963887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.963718    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 13:17:49.963965 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.963711    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 13:17:49.964031 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.963998    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 13:17:49.964592 ip-10-0-136-232 systemd[1]: Started Kubernetes Kubelet.
Jul 23 13:17:49.965210 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.965154    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 13:17:49.965253 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.965223    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-232.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 13:17:49.965350 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.965325    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 13:17:49.965643 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.965602    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 13:17:49.965955 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.965942    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 13:17:49.969732 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.969712    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8f8s6"
Jul 23 13:17:49.969874 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.969858    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 13:17:49.970342 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.969428    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-232.ec2.internal.18c4edaabfcbd31d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-232.ec2.internal,UID:ip-10-0-136-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-232.ec2.internal,},FirstTimestamp:2026-07-23 13:17:49.963588381 +0000 UTC m=+0.339539150,LastTimestamp:2026-07-23 13:17:49.963588381 +0000 UTC m=+0.339539150,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-232.ec2.internal,}"
Jul 23 13:17:49.972353 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.972336    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 13:17:49.972433 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.972356    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 13:17:49.972489 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.972458    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:49.973033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.972994    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 13:17:49.973115 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973029    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 13:17:49.973115 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973016    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 13:17:49.973223 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973141    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 13:17:49.973223 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973149    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 13:17:49.973557 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973519    2559 factory.go:55] Registering systemd factory
Jul 23 13:17:49.973655 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973562    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 23 13:17:49.973786 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973770    2559 factory.go:153] Registering CRI-O factory
Jul 23 13:17:49.973786 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973787    2559 factory.go:223] Registration of the crio container factory successfully
Jul 23 13:17:49.973898 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973831    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 13:17:49.973943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973904    2559 factory.go:103] Registering Raw factory
Jul 23 13:17:49.973943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.973924    2559 manager.go:1196] Started watching for new ooms in manager
Jul 23 13:17:49.974576 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.974563    2559 manager.go:349] Starting recovery of all containers
Jul 23 13:17:49.978197 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.978175    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8f8s6"
Jul 23 13:17:49.978496 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.978466    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 13:17:49.979027 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.978999    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 13:17:49.984782 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.984754    2559 manager.go:354] Recovery completed
Jul 23 13:17:49.988056 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.988028    2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 23 13:17:49.991078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.991062    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:49.992942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.992923    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:49.992996 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.992957    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:49.992996 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.992968    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:49.993568 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.993555    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 13:17:49.993601 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.993568    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:49.993654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.993569    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 13:17:49.993654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.993637    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 13:17:49.994225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.994210    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:49.994281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.994241    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:49.994281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.994257    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:49.994649 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.994636    2559 policy_none.go:49] "None policy: Start"
Jul 23 13:17:49.994691 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.994654    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 13:17:49.994691 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.994664    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 13:17:49.995359 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:49.995269    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-232.ec2.internal.18c4edaac18bbdfb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-232.ec2.internal,UID:ip-10-0-136-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-232.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-232.ec2.internal,},FirstTimestamp:2026-07-23 13:17:49.992943099 +0000 UTC m=+0.368893869,LastTimestamp:2026-07-23 13:17:49.992943099 +0000 UTC m=+0.368893869,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-232.ec2.internal,}"
Jul 23 13:17:49.995707 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:49.995695    2559 policy_none.go:47] "Start"
Jul 23 13:17:50.026271 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026112    2559 manager.go:337] "Starting Device Plugin manager"
Jul 23 13:17:50.026412 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.026396    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 13:17:50.026476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026415    2559 server.go:87] "Starting device plugin registration server"
Jul 23 13:17:50.026682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026668    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 13:17:50.026761 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026683    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 13:17:50.026837 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026819    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 13:17:50.026950 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026906    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 13:17:50.026950 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.026917    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 13:17:50.027551 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.027520    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 13:17:50.027632 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.027587    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.083782 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.083744    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 13:17:50.085025 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.085007    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 13:17:50.085115 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.085051    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 13:17:50.085115 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.085078    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 13:17:50.085209 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.085119    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 13:17:50.087396 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.087369    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 13:17:50.127583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.127514    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:50.128578 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.128562    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:50.128636 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.128593    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:50.128636 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.128603    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:50.128636 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.128627    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.136987 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.136954    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.137075 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.136984    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-232.ec2.internal\": node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.152126 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.152077    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.185631 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.185600    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"]
Jul 23 13:17:50.185690 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.185664    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:50.187156 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.187139    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:50.187233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.187168    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:50.187233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.187178    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:50.188632 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.188619    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:50.188749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.188734    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.188784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.188765    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:50.189401 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.189385    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:50.189476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.189420    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:50.189476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.189429    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:50.189476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.189435    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:50.189476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.189448    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:50.189476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.189457    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:50.190519 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.190506    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.190576 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.190550    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 13:17:50.191239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.191224    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 13:17:50.191314 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.191253    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 13:17:50.191314 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.191266    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeHasSufficientPID"
Jul 23 13:17:50.212113 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.212093    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-232.ec2.internal\" not found" node="ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.216626 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.216611    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-232.ec2.internal\" not found" node="ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.253045 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.253025    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.274415 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.274393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4508d65acded9128b3e84b09c90ba208-config\") pod \"kube-apiserver-proxy-ip-10-0-136-232.ec2.internal\" (UID: \"4508d65acded9128b3e84b09c90ba208\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.274478 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.274420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/32df862deeba2e12946b96a9b91481bf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal\" (UID: \"32df862deeba2e12946b96a9b91481bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.274478 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.274451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32df862deeba2e12946b96a9b91481bf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal\" (UID: \"32df862deeba2e12946b96a9b91481bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.353157 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.353129    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.375520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.375498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4508d65acded9128b3e84b09c90ba208-config\") pod \"kube-apiserver-proxy-ip-10-0-136-232.ec2.internal\" (UID: \"4508d65acded9128b3e84b09c90ba208\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.375620 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.375525    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/32df862deeba2e12946b96a9b91481bf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal\" (UID: \"32df862deeba2e12946b96a9b91481bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.375620 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.375556    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32df862deeba2e12946b96a9b91481bf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal\" (UID: \"32df862deeba2e12946b96a9b91481bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.375620 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.375583    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32df862deeba2e12946b96a9b91481bf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal\" (UID: \"32df862deeba2e12946b96a9b91481bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.375620 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.375611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/32df862deeba2e12946b96a9b91481bf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal\" (UID: \"32df862deeba2e12946b96a9b91481bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.375769 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.375610    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4508d65acded9128b3e84b09c90ba208-config\") pod \"kube-apiserver-proxy-ip-10-0-136-232.ec2.internal\" (UID: \"4508d65acded9128b3e84b09c90ba208\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.453852 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.453775    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.515501 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.515475    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.519561 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.519316    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:50.554377 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.554353    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.654873 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.654838    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.755430 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.755366    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.855950 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.855915    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.913433 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.913410    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 13:17:50.913966 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.913562    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 13:17:50.956033 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:50.956002    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:50.959099 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.959080    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 13:17:50.986988 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.986927    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 13:12:49 +0000 UTC" deadline="2028-03-05 08:28:03.808783925 +0000 UTC"
Jul 23 13:17:50.986988 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:50.986976    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14179h10m12.821812218s"
Jul 23 13:17:51.015893 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:51.015861    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4508d65acded9128b3e84b09c90ba208.slice/crio-eec5f1819701d47138ae4aec040d36580aecf3100cd6b9277a9f78b9624f840a WatchSource:0}: Error finding container eec5f1819701d47138ae4aec040d36580aecf3100cd6b9277a9f78b9624f840a: Status 404 returned error can't find the container with id eec5f1819701d47138ae4aec040d36580aecf3100cd6b9277a9f78b9624f840a
Jul 23 13:17:51.016248 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:51.016218    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32df862deeba2e12946b96a9b91481bf.slice/crio-47ab3f613151777a476020a4420462e646816a61b82ba075c8d6b998787e4381 WatchSource:0}: Error finding container 47ab3f613151777a476020a4420462e646816a61b82ba075c8d6b998787e4381: Status 404 returned error can't find the container with id 47ab3f613151777a476020a4420462e646816a61b82ba075c8d6b998787e4381
Jul 23 13:17:51.021364 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.021350    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 13:17:51.056102 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:51.056069    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:51.059514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.059498    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 13:17:51.088512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.088468    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal" event={"ID":"4508d65acded9128b3e84b09c90ba208","Type":"ContainerStarted","Data":"eec5f1819701d47138ae4aec040d36580aecf3100cd6b9277a9f78b9624f840a"}
Jul 23 13:17:51.089471 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.089449    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal" event={"ID":"32df862deeba2e12946b96a9b91481bf","Type":"ContainerStarted","Data":"47ab3f613151777a476020a4420462e646816a61b82ba075c8d6b998787e4381"}
Jul 23 13:17:51.156625 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:51.156592    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:51.257166 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:51.257136    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-232.ec2.internal\" not found"
Jul 23 13:17:51.259778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.259763    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 13:17:51.272905 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.272863    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:51.286372 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.286355    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 13:17:51.287520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.287509    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal"
Jul 23 13:17:51.294298 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.294285    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 13:17:51.952881 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.952849    2559 apiserver.go:52] "Watching apiserver"
Jul 23 13:17:51.962108 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.962086    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 13:17:51.963777 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.963753    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-vfv67","openshift-network-diagnostics/network-check-target-6bf7k","openshift-network-operator/iptables-alerter-nvgvz","kube-system/global-pull-secret-syncer-z5rhr","kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal","openshift-multus/multus-additional-cni-plugins-6ltzr","openshift-multus/network-metrics-daemon-dn6zg","openshift-ovn-kubernetes/ovnkube-node-qbvpc","kube-system/konnectivity-agent-6tpkf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg","openshift-cluster-node-tuning-operator/tuned-vrq5d","openshift-dns/node-resolver-hvcq5","openshift-image-registry/node-ca-7v5cx"]
Jul 23 13:17:51.965361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.965344    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.966847 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.966831    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.967725 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.967712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:51.968936 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.968921    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:51.969040 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:51.968982    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:17:51.969976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.969957    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:51.971128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.971109    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:51.971204 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:51.971180    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:17:51.972422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.972404    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.973460 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.973333    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 13:17:51.974250 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.974232    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:51.975806 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.975788    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:51.975884 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:51.975844    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:17:51.977202 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977184    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:51.977302 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977275    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 13:17:51.977393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977377    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 13:17:51.977626 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977608    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 13:17:51.978001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977846    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.978001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977895    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 13:17:51.978001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.977924    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.978490 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.978467    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 13:17:51.978600 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.978502    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.978667 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.978602    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vfv67"
Jul 23 13:17:51.978729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.978693    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4xttt\""
Jul 23 13:17:51.978981 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.978952    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 13:17:51.979054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.979036    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.979100 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.979084    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.979657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.979639    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.979790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.979772    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:51.979924 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.979910    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.980064 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980049    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.980179 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980151    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.980179 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980159    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.980318 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980260    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.980348 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980323    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 13:17:51.980412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980395    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 13:17:51.980952 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980930    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-64g2f\""
Jul 23 13:17:51.981035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980950    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 13:17:51.981035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.980977    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cvnl4\""
Jul 23 13:17:51.981160 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.981140    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.981233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.981176    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 13:17:51.981233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.981209    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7jpcf\""
Jul 23 13:17:51.981233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.981227    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9qwct\""
Jul 23 13:17:51.981384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.981367    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wxps5\""
Jul 23 13:17:51.983740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f12762d9-24e8-4482-a8ec-7e294c94cb79-serviceca\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:51.983824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983749    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q5nx\" (UniqueName: \"kubernetes.io/projected/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-kube-api-access-5q5nx\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:51.983824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-socket-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.983824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983778    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-etc-selinux\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.983824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f498563d-5c40-4fdb-b1b1-6b255440eff6-dbus\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:51.983824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983806    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p4pz\" (UniqueName: \"kubernetes.io/projected/e4468347-7f97-4755-8ed3-44c90df023d3-kube-api-access-2p4pz\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:51.983824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-run-netns\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9sxt\" (UniqueName: \"kubernetes.io/projected/81699778-9c7e-42ef-9088-3025a793c2cd-kube-api-access-t9sxt\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983869    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-sys-fs\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzrlg\" (UniqueName: \"kubernetes.io/projected/297549ba-9e31-4efa-a75c-c9c75639f128-kube-api-access-xzrlg\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81699778-9c7e-42ef-9088-3025a793c2cd-ovn-node-metrics-cert\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983908    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-os-release\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983921    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvvb\" (UniqueName: \"kubernetes.io/projected/c93e2ef1-8d3a-4194-ac04-7643f32891a4-kube-api-access-7lvvb\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxhgq\" (UniqueName: \"kubernetes.io/projected/f12762d9-24e8-4482-a8ec-7e294c94cb79-kube-api-access-zxhgq\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983971    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c296489-0fc6-4381-87da-d596fad7eaf1-hosts-file\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.983991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn6h6\" (UniqueName: \"kubernetes.io/projected/4c296489-0fc6-4381-87da-d596fad7eaf1-kube-api-access-tn6h6\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f498563d-5c40-4fdb-b1b1-6b255440eff6-kubelet-config\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-registration-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.984093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-iptables-alerter-script\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-systemd\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-ovnkube-config\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-env-overrides\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984175    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984226    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984245    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-kubelet\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984274    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984295    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-log-socket\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-system-cni-dir\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f12762d9-24e8-4482-a8ec-7e294c94cb79-host\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c296489-0fc6-4381-87da-d596fad7eaf1-tmp-dir\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-device-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-slash\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-cni-bin\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.984683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-ovnkube-script-lib\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984461    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-systemd-units\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-etc-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-ovn\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984553    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-node-log\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-cni-netd\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984612    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cni-binary-copy\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984647    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-host-slash\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984670    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-var-lib-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cnibin\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:51.985341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.984745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:51.990193 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.989766    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 13:17:51.990193 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.990104    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 13:17:51.990697 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.990678    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 13:17:51.990697 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.990688    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 13:17:51.990815 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.990696    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8zqc6\""
Jul 23 13:17:51.991186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.991170    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jnf9n\""
Jul 23 13:17:51.991267 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.991253    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 13:17:51.991327 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:51.991312    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sw8kx\""
Jul 23 13:17:52.002003 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.001986    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 13:17:52.029039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.029022    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rm4h9"
Jul 23 13:17:52.039238 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.039216    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rm4h9"
Jul 23 13:17:52.074275 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.074244    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 13:17:52.085473 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-etc-kubernetes\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.085616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysctl-d\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.085616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysctl-conf\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.085616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.085616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085600    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-kubelet\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085620    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-log-socket\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085637    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-system-cni-dir\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c296489-0fc6-4381-87da-d596fad7eaf1-tmp-dir\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-system-cni-dir\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-log-socket\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-hostroot\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.085773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-sys\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085781    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-tuned\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-ovnkube-script-lib\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-cni-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c296489-0fc6-4381-87da-d596fad7eaf1-tmp-dir\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-tmp\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.085990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkq9r\" (UniqueName: \"kubernetes.io/projected/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-kube-api-access-dkq9r\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-systemd-units\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.086035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-etc-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-systemd-units\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086054    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-etc-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086073    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-node-log\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-kubelet\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-cni-netd\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-cni-netd\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cni-binary-copy\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086143    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-netns\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-kubernetes\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/23df0f51-8206-4026-bdd0-4406375372c9-agent-certs\") pod \"konnectivity-agent-6tpkf\" (UID: \"23df0f51-8206-4026-bdd0-4406375372c9\") " pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-cnibin\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f834fe33-28ca-4c8f-865c-2318f55d0e1a-cni-binary-copy\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086288    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-var-lib-kubelet\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q5nx\" (UniqueName: \"kubernetes.io/projected/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-kube-api-access-5q5nx\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086330    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-socket-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086344    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f498563d-5c40-4fdb-b1b1-6b255440eff6-dbus\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p4pz\" (UniqueName: \"kubernetes.io/projected/e4468347-7f97-4755-8ed3-44c90df023d3-kube-api-access-2p4pz\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-run-netns\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9sxt\" (UniqueName: \"kubernetes.io/projected/81699778-9c7e-42ef-9088-3025a793c2cd-kube-api-access-t9sxt\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086432    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvvb\" (UniqueName: \"kubernetes.io/projected/c93e2ef1-8d3a-4194-ac04-7643f32891a4-kube-api-access-7lvvb\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxhgq\" (UniqueName: \"kubernetes.io/projected/f12762d9-24e8-4482-a8ec-7e294c94cb79-kube-api-access-zxhgq\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-sys-fs\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086481    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-run\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-node-log\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/23df0f51-8206-4026-bdd0-4406375372c9-konnectivity-ca\") pod \"konnectivity-agent-6tpkf\" (UID: \"23df0f51-8206-4026-bdd0-4406375372c9\") " pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086575    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-iptables-alerter-script\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086600    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-ovnkube-config\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086618    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-env-overrides\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-ovnkube-script-lib\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f498563d-5c40-4fdb-b1b1-6b255440eff6-dbus\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-run-netns\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.086809 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.086747    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.086807    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:52.586784375 +0000 UTC m=+2.962735142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086950    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086977    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.086995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f12762d9-24e8-4482-a8ec-7e294c94cb79-host\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cni-binary-copy\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-socket-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087046    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-sys-fs\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f12762d9-24e8-4482-a8ec-7e294c94cb79-host\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-os-release\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.087133    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087167    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-cni-multus\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-lib-modules\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087218    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-device-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087242    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-slash\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087265    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-cni-bin\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087273    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-env-overrides\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-cni-bin\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-modprobe-d\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-ovn\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087425    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087462    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-system-cni-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087480    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-iptables-alerter-script\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087485    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-conf-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-daemon-config\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-host-slash\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087544    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-cni-bin\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-ovn\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087599    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81699778-9c7e-42ef-9088-3025a793c2cd-ovnkube-config\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.087958 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.087633    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:52.587618887 +0000 UTC m=+2.963569667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087650    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-var-lib-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087678    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cnibin\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087684    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-host-slash\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087706    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-multus-certs\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cnibin\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087722    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysconfig\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-device-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087793    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-host-slash\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087796    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-host\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-var-lib-openvswitch\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f12762d9-24e8-4482-a8ec-7e294c94cb79-serviceca\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-systemd\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-etc-selinux\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.088657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzrlg\" (UniqueName: \"kubernetes.io/projected/297549ba-9e31-4efa-a75c-c9c75639f128-kube-api-access-xzrlg\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.087987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81699778-9c7e-42ef-9088-3025a793c2cd-ovn-node-metrics-cert\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-etc-selinux\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088054    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-os-release\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c296489-0fc6-4381-87da-d596fad7eaf1-hosts-file\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6h6\" (UniqueName: \"kubernetes.io/projected/4c296489-0fc6-4381-87da-d596fad7eaf1-kube-api-access-tn6h6\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c296489-0fc6-4381-87da-d596fad7eaf1-hosts-file\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-socket-dir-parent\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088244    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c93e2ef1-8d3a-4194-ac04-7643f32891a4-os-release\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088257    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-kubelet\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f12762d9-24e8-4482-a8ec-7e294c94cb79-serviceca\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zxf\" (UniqueName: \"kubernetes.io/projected/f834fe33-28ca-4c8f-865c-2318f55d0e1a-kube-api-access-g7zxf\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088327    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f498563d-5c40-4fdb-b1b1-6b255440eff6-kubelet-config\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088345    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088373    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c93e2ef1-8d3a-4194-ac04-7643f32891a4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f498563d-5c40-4fdb-b1b1-6b255440eff6-kubelet-config\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-registration-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.089322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-systemd\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.090023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/297549ba-9e31-4efa-a75c-c9c75639f128-registration-dir\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.090023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-k8s-cni-cncf-io\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.090023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.088472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81699778-9c7e-42ef-9088-3025a793c2cd-run-systemd\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.091652 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.091635    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81699778-9c7e-42ef-9088-3025a793c2cd-ovn-node-metrics-cert\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.099058 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.099009    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvvb\" (UniqueName: \"kubernetes.io/projected/c93e2ef1-8d3a-4194-ac04-7643f32891a4-kube-api-access-7lvvb\") pod \"multus-additional-cni-plugins-6ltzr\" (UID: \"c93e2ef1-8d3a-4194-ac04-7643f32891a4\") " pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.100169 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.100128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzrlg\" (UniqueName: \"kubernetes.io/projected/297549ba-9e31-4efa-a75c-c9c75639f128-kube-api-access-xzrlg\") pod \"aws-ebs-csi-driver-node-rl9rg\" (UID: \"297549ba-9e31-4efa-a75c-c9c75639f128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.100626 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.100597    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn6h6\" (UniqueName: \"kubernetes.io/projected/4c296489-0fc6-4381-87da-d596fad7eaf1-kube-api-access-tn6h6\") pod \"node-resolver-hvcq5\" (UID: \"4c296489-0fc6-4381-87da-d596fad7eaf1\") " pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.100687 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.100624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p4pz\" (UniqueName: \"kubernetes.io/projected/e4468347-7f97-4755-8ed3-44c90df023d3-kube-api-access-2p4pz\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:52.101277 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.101256    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxhgq\" (UniqueName: \"kubernetes.io/projected/f12762d9-24e8-4482-a8ec-7e294c94cb79-kube-api-access-zxhgq\") pod \"node-ca-7v5cx\" (UID: \"f12762d9-24e8-4482-a8ec-7e294c94cb79\") " pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.101520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.101495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q5nx\" (UniqueName: \"kubernetes.io/projected/7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102-kube-api-access-5q5nx\") pod \"iptables-alerter-nvgvz\" (UID: \"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102\") " pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.101657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.101556    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9sxt\" (UniqueName: \"kubernetes.io/projected/81699778-9c7e-42ef-9088-3025a793c2cd-kube-api-access-t9sxt\") pod \"ovnkube-node-qbvpc\" (UID: \"81699778-9c7e-42ef-9088-3025a793c2cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.188941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188859    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-socket-dir-parent\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.188941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-kubelet\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.188941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zxf\" (UniqueName: \"kubernetes.io/projected/f834fe33-28ca-4c8f-865c-2318f55d0e1a-kube-api-access-g7zxf\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.188941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-k8s-cni-cncf-io\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-etc-kubernetes\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysctl-d\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.188991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysctl-conf\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-hostroot\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-sys\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-tuned\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-cni-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-tmp\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkq9r\" (UniqueName: \"kubernetes.io/projected/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-kube-api-access-dkq9r\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189131    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-netns\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189152    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-kubernetes\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/23df0f51-8206-4026-bdd0-4406375372c9-agent-certs\") pod \"konnectivity-agent-6tpkf\" (UID: \"23df0f51-8206-4026-bdd0-4406375372c9\") " pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.189216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189208    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-cnibin\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f834fe33-28ca-4c8f-865c-2318f55d0e1a-cni-binary-copy\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-var-lib-kubelet\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189288    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-run\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/23df0f51-8206-4026-bdd0-4406375372c9-konnectivity-ca\") pod \"konnectivity-agent-6tpkf\" (UID: \"23df0f51-8206-4026-bdd0-4406375372c9\") " pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-os-release\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-cni-multus\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-lib-modules\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-cni-bin\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-modprobe-d\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-system-cni-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-conf-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-daemon-config\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189613    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-multus-certs\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysconfig\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-os-release\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189675    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-host\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.189794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-systemd\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-kubernetes\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.189781    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-run\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190196    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f834fe33-28ca-4c8f-865c-2318f55d0e1a-cni-binary-copy\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190200    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-daemon-config\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190241    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-cni-multus\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-socket-dir-parent\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-kubelet\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-lib-modules\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190518    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-var-lib-kubelet\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-hostroot\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-var-lib-cni-bin\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190696    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-modprobe-d\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190781    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/23df0f51-8206-4026-bdd0-4406375372c9-konnectivity-ca\") pod \"konnectivity-agent-6tpkf\" (UID: \"23df0f51-8206-4026-bdd0-4406375372c9\") " pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-sys\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190839    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-netns\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-multus-certs\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190876    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-cni-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190900    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-host\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190913    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-system-cni-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.190993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.190949    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-multus-conf-dir\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysconfig\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191060    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysctl-conf\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191103    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-systemd\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191171    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-cnibin\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-host-run-k8s-cni-cncf-io\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191219    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f834fe33-28ca-4c8f-865c-2318f55d0e1a-etc-kubernetes\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.191445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.191322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-sysctl-d\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.193140 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.193120    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-etc-tuned\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.193289 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.193211    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-tmp\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.193343 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.193311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/23df0f51-8206-4026-bdd0-4406375372c9-agent-certs\") pod \"konnectivity-agent-6tpkf\" (UID: \"23df0f51-8206-4026-bdd0-4406375372c9\") " pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.213808 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.213789    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:17:52.213911 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.213821    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:17:52.213911 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.213834    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:52.213911 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.213900    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:52.713883808 +0000 UTC m=+3.089834578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:52.215956 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.215937    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkq9r\" (UniqueName: \"kubernetes.io/projected/44dae05c-0d4b-4cba-9577-2f7c5753dd3d-kube-api-access-dkq9r\") pod \"tuned-vrq5d\" (UID: \"44dae05c-0d4b-4cba-9577-2f7c5753dd3d\") " pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.216931 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.216916    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zxf\" (UniqueName: \"kubernetes.io/projected/f834fe33-28ca-4c8f-865c-2318f55d0e1a-kube-api-access-g7zxf\") pod \"multus-vfv67\" (UID: \"f834fe33-28ca-4c8f-865c-2318f55d0e1a\") " pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.276195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.276166    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6ltzr"
Jul 23 13:17:52.283128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.283109    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg"
Jul 23 13:17:52.291830 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.291812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nvgvz"
Jul 23 13:17:52.303053 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.303033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:17:52.304068 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.304045    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7v5cx"
Jul 23 13:17:52.309300 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.309280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hvcq5"
Jul 23 13:17:52.315788 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.315772    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:17:52.321457 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.321440    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vfv67"
Jul 23 13:17:52.325564 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.325545    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vrq5d"
Jul 23 13:17:52.443838 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.443764    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 13:17:52.520060 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.520034    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12762d9_24e8_4482_a8ec_7e294c94cb79.slice/crio-a26b4f1336551438c90f27812b5d5c3b5469e67504b0e0e737e9ab75fd8cf429 WatchSource:0}: Error finding container a26b4f1336551438c90f27812b5d5c3b5469e67504b0e0e737e9ab75fd8cf429: Status 404 returned error can't find the container with id a26b4f1336551438c90f27812b5d5c3b5469e67504b0e0e737e9ab75fd8cf429
Jul 23 13:17:52.533274 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.533248    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5b16fb_26ed_4b8f_ad2b_cdeeba8b3102.slice/crio-2014eda553094d3ba82c4204c3ff729f18d8d9cafd5bd00c1ba30646feef13cc WatchSource:0}: Error finding container 2014eda553094d3ba82c4204c3ff729f18d8d9cafd5bd00c1ba30646feef13cc: Status 404 returned error can't find the container with id 2014eda553094d3ba82c4204c3ff729f18d8d9cafd5bd00c1ba30646feef13cc
Jul 23 13:17:52.538829 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.538806    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23df0f51_8206_4026_bdd0_4406375372c9.slice/crio-d78672154c585215e96aebe5045aa6fc01e345b3334285758bb2ad096f3721b2 WatchSource:0}: Error finding container d78672154c585215e96aebe5045aa6fc01e345b3334285758bb2ad096f3721b2: Status 404 returned error can't find the container with id d78672154c585215e96aebe5045aa6fc01e345b3334285758bb2ad096f3721b2
Jul 23 13:17:52.540643 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.540622    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297549ba_9e31_4efa_a75c_c9c75639f128.slice/crio-647edad36e1b072b1d4c6b8781e10da7e1dfe083828e189b2330711d1f666f5b WatchSource:0}: Error finding container 647edad36e1b072b1d4c6b8781e10da7e1dfe083828e189b2330711d1f666f5b: Status 404 returned error can't find the container with id 647edad36e1b072b1d4c6b8781e10da7e1dfe083828e189b2330711d1f666f5b
Jul 23 13:17:52.541187 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.541164    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf834fe33_28ca_4c8f_865c_2318f55d0e1a.slice/crio-55f2f976d1b55967405161a3e1cfd8108dfc5e82cc1c54b520c1840ad9677db5 WatchSource:0}: Error finding container 55f2f976d1b55967405161a3e1cfd8108dfc5e82cc1c54b520c1840ad9677db5: Status 404 returned error can't find the container with id 55f2f976d1b55967405161a3e1cfd8108dfc5e82cc1c54b520c1840ad9677db5
Jul 23 13:17:52.542810 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.542792    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44dae05c_0d4b_4cba_9577_2f7c5753dd3d.slice/crio-61c2c5bf138d4e91eef47e267aae7de960eb86f5c7abb181071c705f2c33cb9e WatchSource:0}: Error finding container 61c2c5bf138d4e91eef47e267aae7de960eb86f5c7abb181071c705f2c33cb9e: Status 404 returned error can't find the container with id 61c2c5bf138d4e91eef47e267aae7de960eb86f5c7abb181071c705f2c33cb9e
Jul 23 13:17:52.543553 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.543502    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93e2ef1_8d3a_4194_ac04_7643f32891a4.slice/crio-16c8a9d1c2ef35851ff28c3beb1c09c7e05cd2bc760c892a6ebf910d4acda19c WatchSource:0}: Error finding container 16c8a9d1c2ef35851ff28c3beb1c09c7e05cd2bc760c892a6ebf910d4acda19c: Status 404 returned error can't find the container with id 16c8a9d1c2ef35851ff28c3beb1c09c7e05cd2bc760c892a6ebf910d4acda19c
Jul 23 13:17:52.545257 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.544314    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81699778_9c7e_42ef_9088_3025a793c2cd.slice/crio-16d3e64b1b2254bb610df589f1485aaffccd45a5c38055afa4749d44ede50be2 WatchSource:0}: Error finding container 16d3e64b1b2254bb610df589f1485aaffccd45a5c38055afa4749d44ede50be2: Status 404 returned error can't find the container with id 16d3e64b1b2254bb610df589f1485aaffccd45a5c38055afa4749d44ede50be2
Jul 23 13:17:52.545601 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:17:52.545576    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c296489_0fc6_4381_87da_d596fad7eaf1.slice/crio-dd90a30023b8dc3671e894c270521bab817466f3bfc047fc176b3711ed589d18 WatchSource:0}: Error finding container dd90a30023b8dc3671e894c270521bab817466f3bfc047fc176b3711ed589d18: Status 404 returned error can't find the container with id dd90a30023b8dc3671e894c270521bab817466f3bfc047fc176b3711ed589d18
Jul 23 13:17:52.593492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.593470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:52.593611 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.593524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:52.593654 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.593616    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:52.593654 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.593619    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:52.593720 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.593664    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:53.593652999 +0000 UTC m=+3.969603753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:52.593720 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.593677    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:53.593670057 +0000 UTC m=+3.969620811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:52.795308 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:52.795225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:52.795459 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.795360    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:17:52.795459 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.795377    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:17:52.795459 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.795387    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:52.795459 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:52.795458    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:53.795426466 +0000 UTC m=+4.171377225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:53.041218 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.041128    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 13:12:51 +0000 UTC" deadline="2028-02-15 14:42:01.249574244 +0000 UTC"
Jul 23 13:17:53.041218 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.041165    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13729h24m8.208413104s"
Jul 23 13:17:53.101210 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.101060    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hvcq5" event={"ID":"4c296489-0fc6-4381-87da-d596fad7eaf1","Type":"ContainerStarted","Data":"dd90a30023b8dc3671e894c270521bab817466f3bfc047fc176b3711ed589d18"}
Jul 23 13:17:53.106188 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.106079    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"16d3e64b1b2254bb610df589f1485aaffccd45a5c38055afa4749d44ede50be2"}
Jul 23 13:17:53.113429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.113348    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg" event={"ID":"297549ba-9e31-4efa-a75c-c9c75639f128","Type":"ContainerStarted","Data":"647edad36e1b072b1d4c6b8781e10da7e1dfe083828e189b2330711d1f666f5b"}
Jul 23 13:17:53.120654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.120629    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6tpkf" event={"ID":"23df0f51-8206-4026-bdd0-4406375372c9","Type":"ContainerStarted","Data":"d78672154c585215e96aebe5045aa6fc01e345b3334285758bb2ad096f3721b2"}
Jul 23 13:17:53.127251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.127226    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vrq5d" event={"ID":"44dae05c-0d4b-4cba-9577-2f7c5753dd3d","Type":"ContainerStarted","Data":"61c2c5bf138d4e91eef47e267aae7de960eb86f5c7abb181071c705f2c33cb9e"}
Jul 23 13:17:53.135307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.135280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vfv67" event={"ID":"f834fe33-28ca-4c8f-865c-2318f55d0e1a","Type":"ContainerStarted","Data":"55f2f976d1b55967405161a3e1cfd8108dfc5e82cc1c54b520c1840ad9677db5"}
Jul 23 13:17:53.144773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.144743    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerStarted","Data":"16c8a9d1c2ef35851ff28c3beb1c09c7e05cd2bc760c892a6ebf910d4acda19c"}
Jul 23 13:17:53.155380 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.155346    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nvgvz" event={"ID":"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102","Type":"ContainerStarted","Data":"2014eda553094d3ba82c4204c3ff729f18d8d9cafd5bd00c1ba30646feef13cc"}
Jul 23 13:17:53.169947 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.169908    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7v5cx" event={"ID":"f12762d9-24e8-4482-a8ec-7e294c94cb79","Type":"ContainerStarted","Data":"a26b4f1336551438c90f27812b5d5c3b5469e67504b0e0e737e9ab75fd8cf429"}
Jul 23 13:17:53.186255 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.186185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal" event={"ID":"4508d65acded9128b3e84b09c90ba208","Type":"ContainerStarted","Data":"46ced03b0e8229d49f3a3fa041174d6874f99209e587e9e9dcd2ea9c13704d01"}
Jul 23 13:17:53.200162 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.200092    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-232.ec2.internal" podStartSLOduration=3.200010271 podStartE2EDuration="3.200010271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:17:53.199734088 +0000 UTC m=+3.575684857" watchObservedRunningTime="2026-07-23 13:17:53.200010271 +0000 UTC m=+3.575961045"
Jul 23 13:17:53.603965 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.603411    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:53.603965 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.603777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:53.603965 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.603916    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:53.603965 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.603975    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:55.603959434 +0000 UTC m=+5.979910207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:53.604390 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.604346    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:53.604494 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.604400    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:55.60438528 +0000 UTC m=+5.980336034 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:53.806665 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:53.806514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:53.812951 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.812777    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:17:53.812951 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.812809    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:17:53.812951 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.812821    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:53.812951 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:53.812908    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:55.812885512 +0000 UTC m=+6.188836281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:54.041589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:54.041503    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 13:12:51 +0000 UTC" deadline="2028-03-10 14:19:10.620237434 +0000 UTC"
Jul 23 13:17:54.041589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:54.041583    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14305h1m16.57865843s"
Jul 23 13:17:54.087690 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:54.087591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:54.087844 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:54.087705    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:17:54.088114 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:54.088095    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:54.088198 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:54.088181    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:17:54.088264 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:54.088249    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:54.088352 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:54.088334    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:17:55.621568 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:55.621514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:55.621977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:55.621591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:55.621977 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.621744    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:55.621977 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.621804    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:59.621785954 +0000 UTC m=+9.997736709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:55.622193 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.622176    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:55.622261 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.622229    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:59.622214573 +0000 UTC m=+9.998165343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:55.824161 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:55.824122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:55.824318 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.824289    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:17:55.824318 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.824308    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:17:55.824318 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.824319    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:55.824477 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:55.824377    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:17:59.824358116 +0000 UTC m=+10.200308873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:56.086340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:56.086270    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:56.086340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:56.086288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:56.086569 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:56.086400    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:17:56.086569 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:56.086510    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:17:56.086680 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:56.086583    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:56.086680 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:56.086652    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:17:58.087310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:58.086635    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:58.087310 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:58.086769    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:17:58.087310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:58.087151    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:58.087310 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:58.087254    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:17:58.087948 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:58.087414    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:58.087948 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:58.087524    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:17:59.661084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:59.661041    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:17:59.661550 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:59.661101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:17:59.661550 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.661199    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:59.661550 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.661206    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:59.661550 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.661259    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:07.661246119 +0000 UTC m=+18.037196875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:17:59.661550 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.661277    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:07.661267321 +0000 UTC m=+18.037218089 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:17:59.863961 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:17:59.863421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:17:59.863961 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.863592    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:17:59.863961 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.863609    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:17:59.863961 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.863618    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:17:59.863961 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:17:59.863668    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:07.863655452 +0000 UTC m=+18.239606208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:18:00.087191 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:00.087107    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:00.087325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:00.087227    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:00.087325 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:00.087249    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:00.087409 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:00.087324    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:00.087409 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:00.087371    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:00.087478 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:00.087420    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:02.085384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:02.085350    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:02.085384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:02.085388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:02.085899 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:02.085356    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:02.085899 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:02.085483    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:02.085899 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:02.085611    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:02.085899 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:02.085689    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:04.086756 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:04.086300    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:04.086756 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:04.086316    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:04.086756 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:04.086434    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:04.086756 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:04.086430    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:04.086756 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:04.086571    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:04.086756 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:04.086654    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:06.086330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:06.086293    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:06.086753 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:06.086293    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:06.086753 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:06.086440    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:06.086753 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:06.086512    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:06.086753 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:06.086299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:06.086753 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:06.086626    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:07.721493 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:07.721455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:07.721493 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:07.721500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:07.722074 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.721639    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:18:07.722074 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.721639    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:18:07.722074 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.721707    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:23.721690482 +0000 UTC m=+34.097641256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:18:07.722074 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.721722    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:23.72171534 +0000 UTC m=+34.097666096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:18:07.922831 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:07.922791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:07.922999 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.922982    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:18:07.923046 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.923009    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:18:07.923046 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.923020    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:18:07.923115 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:07.923081    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:23.923055967 +0000 UTC m=+34.299006722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:18:08.085922 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:08.085835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:08.085922 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:08.085865    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:08.086138 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:08.085960    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:08.086138 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:08.086070    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:08.086232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:08.086153    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:08.086283 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:08.086254    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:10.086462 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:10.086430    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:10.086917 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:10.086514    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:10.086917 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:10.086548    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:10.086917 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:10.086664    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:10.086917 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:10.086699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:10.086917 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:10.086776    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:12.085616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.085368    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:12.086245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.085383    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:12.086245 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:12.085727    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:12.086245 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:12.085789    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:12.086245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.085387    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:12.086245 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:12.085857    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:12.230599 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.230573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"583e8953675cb154af6d22f36bd9eae445dd309303948e6612dde7fbfa562ade"}
Jul 23 13:18:12.230715 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.230661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"de14f6cbc885e24cf5483b601c6f2f4d16246cd96fbff8ee06179be7bf986096"}
Jul 23 13:18:12.230833 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.230716    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"ae3b6838f77bb6ffff09de9333e4c50e51504ce5622cdb256644556dd2677baf"}
Jul 23 13:18:12.230833 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.230763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"216758480b5aa0269eb2a6578ac7407c42a9a6475231260ef3f9b6585047d13f"}
Jul 23 13:18:12.232111 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.232084    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vrq5d" event={"ID":"44dae05c-0d4b-4cba-9577-2f7c5753dd3d","Type":"ContainerStarted","Data":"3c2d64a81da72e599eb789b4235b0aa52df2f9da4e6c1f5974a58786197db8c3"}
Jul 23 13:18:12.233546 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.233505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vfv67" event={"ID":"f834fe33-28ca-4c8f-865c-2318f55d0e1a","Type":"ContainerStarted","Data":"7a9851f18b24490bba4ccd331fb7f523e2caabcb406698fbac88614f13c51861"}
Jul 23 13:18:12.252858 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.252817    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vrq5d" podStartSLOduration=4.068569039 podStartE2EDuration="23.252803465s" totalImagesPullingTime="19.184234426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:12.25268768 +0000 UTC m=+22.628638458" watchObservedRunningTime="2026-07-23 13:18:12.252803465 +0000 UTC m=+22.628754242"
Jul 23 13:18:12.270714 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:12.270674    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vfv67" podStartSLOduration=4.051310841 podStartE2EDuration="23.270657738s" totalImagesPullingTime="19.219346897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:12.270437843 +0000 UTC m=+22.646388643" watchObservedRunningTime="2026-07-23 13:18:12.270657738 +0000 UTC m=+22.646608515"
Jul 23 13:18:13.238148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.237897    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg" event={"ID":"297549ba-9e31-4efa-a75c-c9c75639f128","Type":"ContainerStarted","Data":"d628a7b40e9201cf768bfd5f1962ef7b6c9801042c2e4986f5c1c08a90a256ae"}
Jul 23 13:18:13.239541 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.239509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6tpkf" event={"ID":"23df0f51-8206-4026-bdd0-4406375372c9","Type":"ContainerStarted","Data":"44267cecc23bb1cdfce0767235c4bc53737f7b5a265fdcdfa24a745c4cf0d0f4"}
Jul 23 13:18:13.241278 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.241252    2559 generic.go:356] "Generic (PLEG): container finished" podID="c93e2ef1-8d3a-4194-ac04-7643f32891a4" containerID="9df483a77f4a005ebb490d378c64f6f0f256936e27d2faf2a175b7e3927d838e" exitCode=0
Jul 23 13:18:13.241398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.241334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerDied","Data":"9df483a77f4a005ebb490d378c64f6f0f256936e27d2faf2a175b7e3927d838e"}
Jul 23 13:18:13.243619 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.243594    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nvgvz" event={"ID":"7b5b16fb-26ed-4b8f-ad2b-cdeeba8b3102","Type":"ContainerStarted","Data":"310c8e522b1daf5402029361883ff50e75a3391ecf55f8b28f97f411609e7fc0"}
Jul 23 13:18:13.245221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.245201    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7v5cx" event={"ID":"f12762d9-24e8-4482-a8ec-7e294c94cb79","Type":"ContainerStarted","Data":"03f62d25d7d3b86d269f117eb9fcab7c031418deb0636c37adda8a0da60d4d53"}
Jul 23 13:18:13.248799 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.247912    2559 generic.go:356] "Generic (PLEG): container finished" podID="32df862deeba2e12946b96a9b91481bf" containerID="2d3222d8a24289ee02af007e37df6d9270d12214d6b9e11951584d33169ac36d" exitCode=0
Jul 23 13:18:13.248799 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.248166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal" event={"ID":"32df862deeba2e12946b96a9b91481bf","Type":"ContainerDied","Data":"2d3222d8a24289ee02af007e37df6d9270d12214d6b9e11951584d33169ac36d"}
Jul 23 13:18:13.250723 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.250698    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hvcq5" event={"ID":"4c296489-0fc6-4381-87da-d596fad7eaf1","Type":"ContainerStarted","Data":"8c0c14d93a41878b6352ccacf70c849211accaff6f79470975166863af3808ed"}
Jul 23 13:18:13.253904 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.253880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"eec7a4eb4c232b79ef78cf1319e319079369d7314eae6d057717d03d956c65a3"}
Jul 23 13:18:13.254002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.253965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"eba19d429723ef1c0d849233f5300368f5296b6a8b47d6b85cc9aaed0b44517d"}
Jul 23 13:18:13.279392 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.279344    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nvgvz" podStartSLOduration=5.108981499 podStartE2EDuration="24.279330829s" totalImagesPullingTime="19.17034933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:13.279317871 +0000 UTC m=+23.655268650" watchObservedRunningTime="2026-07-23 13:18:13.279330829 +0000 UTC m=+23.655281606"
Jul 23 13:18:13.279883 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.279842    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6tpkf" podStartSLOduration=13.244870222 podStartE2EDuration="24.279830555s" totalImagesPullingTime="11.034960333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:13.257783355 +0000 UTC m=+23.633734133" watchObservedRunningTime="2026-07-23 13:18:13.279830555 +0000 UTC m=+23.655781336"
Jul 23 13:18:13.300136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.300083    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7v5cx" podStartSLOduration=5.125961739 podStartE2EDuration="24.300066246s" totalImagesPullingTime="19.174104507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:13.299984663 +0000 UTC m=+23.675935441" watchObservedRunningTime="2026-07-23 13:18:13.300066246 +0000 UTC m=+23.676017008"
Jul 23 13:18:13.383442 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.383395    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hvcq5" podStartSLOduration=5.22637855 podStartE2EDuration="24.383379758s" totalImagesPullingTime="19.157001208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:13.383319651 +0000 UTC m=+23.759270430" watchObservedRunningTime="2026-07-23 13:18:13.383379758 +0000 UTC m=+23.759330535"
Jul 23 13:18:13.680505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.680479    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:18:13.681092 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.681075    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:18:13.703896 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.703875    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 13:18:13.727084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.727044    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:13.807593 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:13.807559    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:14.042757 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.042569    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T13:18:13.703891382Z","UUID":"d61795cc-40cb-4804-ab1a-d6f722a39674","Handler":null,"Name":"","Endpoint":""}
Jul 23 13:18:14.044269 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.044246    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 13:18:14.044382 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.044293    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 13:18:14.086408 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.086368    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:14.086605 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.086378    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:14.086605 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:14.086509    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:14.086605 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.086521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:14.086820 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:14.086651    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:14.086820 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:14.086719    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:14.259805 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.259732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal" event={"ID":"32df862deeba2e12946b96a9b91481bf","Type":"ContainerStarted","Data":"2bd1708e407d3175b05f3dafb6ad704ea58935bdbed8aade3b2c81d747b85f36"}
Jul 23 13:18:14.262622 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.262588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg" event={"ID":"297549ba-9e31-4efa-a75c-c9c75639f128","Type":"ContainerStarted","Data":"eff56151864b4ba6ef4241bbd7217154815e5faf0de904e93f2b82bd5d102d20"}
Jul 23 13:18:14.263990 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.263956    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:18:14.264489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.264471    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6tpkf"
Jul 23 13:18:14.276349 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.276303    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-232.ec2.internal" podStartSLOduration=24.276287718 podStartE2EDuration="24.276287718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:14.276150416 +0000 UTC m=+24.652101195" watchObservedRunningTime="2026-07-23 13:18:14.276287718 +0000 UTC m=+24.652238497"
Jul 23 13:18:14.345969 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:14.345888    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:15.268431 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:15.268339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"aacaa4abb16f1224275306a7912655fdf5fa1183a9a546b1bb3b03bc59fa84b5"}
Jul 23 13:18:15.270711 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:15.270682    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg" event={"ID":"297549ba-9e31-4efa-a75c-c9c75639f128","Type":"ContainerStarted","Data":"d226a818835c92eb28d7f9e2b9bfd787cf55001ec7621fceaf6bdc5fb5b62ad5"}
Jul 23 13:18:15.289455 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:15.289410    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rl9rg" podStartSLOduration=4.569467439 podStartE2EDuration="26.289395212s" totalImagesPullingTime="21.719927773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:15.288953832 +0000 UTC m=+25.664904611" watchObservedRunningTime="2026-07-23 13:18:15.289395212 +0000 UTC m=+25.665345990"
Jul 23 13:18:16.086151 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:16.086104    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:16.086334 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:16.086165    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:16.086334 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:16.086179    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:16.086334 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:16.086273    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:16.086653 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:16.086619    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:16.086760 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:16.086712    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:17.071044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:17.071005    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:17.173232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:17.173140    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:18.086562 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.086322    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:18.087314 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.086334    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:18.087314 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:18.086610    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:18.087314 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.086376    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:18.087314 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:18.086688    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:18.087314 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:18.086733    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:18.280420 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.280362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" event={"ID":"81699778-9c7e-42ef-9088-3025a793c2cd","Type":"ContainerStarted","Data":"c96823174a6a79c88cc27bb46607e94ec0d20999d61fcdeba8e26ae9843caf1d"}
Jul 23 13:18:18.280651 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.280635    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:18:18.280715 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.280697    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:18:18.282403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.282374    2559 generic.go:356] "Generic (PLEG): container finished" podID="c93e2ef1-8d3a-4194-ac04-7643f32891a4" containerID="7a134bfb439558f7421315b01dee3f79776384628bbe93d533bb8b36cccde2d4" exitCode=0
Jul 23 13:18:18.282583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.282415    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerDied","Data":"7a134bfb439558f7421315b01dee3f79776384628bbe93d533bb8b36cccde2d4"}
Jul 23 13:18:18.298276 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.298255    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:18:18.298442 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.298427    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:18:18.316908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:18.316853    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc" podStartSLOduration=10.094547771 podStartE2EDuration="29.316840416s" totalImagesPullingTime="19.222292645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:18.316200122 +0000 UTC m=+28.692150900" watchObservedRunningTime="2026-07-23 13:18:18.316840416 +0000 UTC m=+28.692791193"
Jul 23 13:18:19.284438 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.284278    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 23 13:18:19.417403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.417378    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z5rhr"]
Jul 23 13:18:19.417520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.417507    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:19.417642 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:19.417621    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:19.419860 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.419838    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dn6zg"]
Jul 23 13:18:19.419966 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.419930    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:19.420039 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:19.420021    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:19.422028 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.421980    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6bf7k"]
Jul 23 13:18:19.422109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.422053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:19.422159 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:19.422115    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:19.460079 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:19.460051    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:18:20.288480 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:20.288449    2559 generic.go:356] "Generic (PLEG): container finished" podID="c93e2ef1-8d3a-4194-ac04-7643f32891a4" containerID="dcf62225d7003ac2b7d9ee4174643bbe00ac406c09ec7360b9ccb8342c8cbe47" exitCode=0
Jul 23 13:18:20.289011 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:20.288549    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerDied","Data":"dcf62225d7003ac2b7d9ee4174643bbe00ac406c09ec7360b9ccb8342c8cbe47"}
Jul 23 13:18:20.983025 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:20.982982    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:21.048299 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:21.048269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:21.085283 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:21.085257    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:21.085427 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:21.085357    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:21.085427 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:21.085360    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:21.085427 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:21.085367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:21.085610 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:21.085457    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:21.085610 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:21.085564    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:22.284130 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.283924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:22.295442 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.295413    2559 generic.go:356] "Generic (PLEG): container finished" podID="c93e2ef1-8d3a-4194-ac04-7643f32891a4" containerID="b112db848581b0374b406ad78023155a7d6f0ee05663bf6ae1a257258613cc1e" exitCode=0
Jul 23 13:18:22.295590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.295485    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerDied","Data":"b112db848581b0374b406ad78023155a7d6f0ee05663bf6ae1a257258613cc1e"}
Jul 23 13:18:22.346354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.346326    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:22.536710 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.536619    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:22.718221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.718190    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:22.798965 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:22.798890    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:23.086148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.086071    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:23.086289 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.086183    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:23.086289 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.086206    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:23.086289 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.086199    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:23.086289 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.086282    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:23.086438 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.086418    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:23.507161 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.507135    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hvcq5_4c296489-0fc6-4381-87da-d596fad7eaf1/dns-node-resolver/0.log"
Jul 23 13:18:23.623295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.623215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:23.744797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.744760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:23.744797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.744803    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:23.744992 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.744919    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 13:18:23.744992 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.744976    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret podName:f498563d-5c40-4fdb-b1b1-6b255440eff6 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:55.744961994 +0000 UTC m=+66.120912750 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret") pod "global-pull-secret-syncer-z5rhr" (UID: "f498563d-5c40-4fdb-b1b1-6b255440eff6") : object "kube-system"/"original-pull-secret" not registered
Jul 23 13:18:23.745068 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.744919    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:18:23.745068 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.745040    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs podName:e4468347-7f97-4755-8ed3-44c90df023d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:55.745028496 +0000 UTC m=+66.120979252 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs") pod "network-metrics-daemon-dn6zg" (UID: "e4468347-7f97-4755-8ed3-44c90df023d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 13:18:23.946349 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:23.946263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:23.946494 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.946419    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 13:18:23.946494 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.946439    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 13:18:23.946494 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.946448    2559 projected.go:196] Error preparing data for projected volume kube-api-access-kjlx6 for pod openshift-network-diagnostics/network-check-target-6bf7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:18:23.946625 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:23.946497    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6 podName:bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:55.946483712 +0000 UTC m=+66.322434468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kjlx6" (UniqueName: "kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6") pod "network-check-target-6bf7k" (UID: "bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 13:18:24.063817 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:24.063790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:24.175870 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:24.175836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:24.245068 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:24.245002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:24.308470 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:24.308441    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7v5cx_f12762d9-24e8-4482-a8ec-7e294c94cb79/node-ca/0.log"
Jul 23 13:18:24.403579 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:24.403551    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:24.504599 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:24.504497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:25.085582 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.085554    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:25.085962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.085656    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:25.085962 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.085662    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6bf7k" podUID="bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481"
Jul 23 13:18:25.085962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.085691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:25.085962 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.085788    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z5rhr" podUID="f498563d-5c40-4fdb-b1b1-6b255440eff6"
Jul 23 13:18:25.085962 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.085857    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dn6zg" podUID="e4468347-7f97-4755-8ed3-44c90df023d3"
Jul 23 13:18:25.442448 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.442371    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-232.ec2.internal" event="NodeReady"
Jul 23 13:18:25.442619 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.442559    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 13:18:25.485725 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.485694    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f9b69dbf-p46zn"]
Jul 23 13:18:25.487691 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.487670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.490772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.490750    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 13:18:25.491051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.491034    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 13:18:25.491512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.491491    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 13:18:25.491632 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.491559    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cmxhx"]
Jul 23 13:18:25.492164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.492149    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnsmp\""
Jul 23 13:18:25.493846 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.493413    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.495073 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.495051    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mv4jn"]
Jul 23 13:18:25.496650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.496625    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 13:18:25.496908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.496891    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 13:18:25.497051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.497032    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 13:18:25.497452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.497159    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vwfmr\""
Jul 23 13:18:25.497452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.497242    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 13:18:25.499288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.499234    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.499385 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.499347    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 13:18:25.501627 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.501611    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 13:18:25.501799 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.501705    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m94bd\""
Jul 23 13:18:25.501875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.501813    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 13:18:25.507079 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.507058    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f9b69dbf-p46zn"]
Jul 23 13:18:25.511555 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.511515    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mv4jn"]
Jul 23 13:18:25.520029 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.519994    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l5d5t"]
Jul 23 13:18:25.522874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.522853    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmxhx"]
Jul 23 13:18:25.522972 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.522956    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.526549 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.526392    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 13:18:25.526701 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.526680    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 13:18:25.526810 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.526721    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 13:18:25.526810 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.526802    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-85swg\""
Jul 23 13:18:25.535036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.535012    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l5d5t"]
Jul 23 13:18:25.638268 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.638233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:25.659213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659182    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-installation-pull-secrets\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659374 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4vl\" (UniqueName: \"kubernetes.io/projected/1ea1d47b-fbc5-49b3-86d3-26397e143e41-kube-api-access-nz4vl\") pod \"ingress-canary-l5d5t\" (UID: \"1ea1d47b-fbc5-49b3-86d3-26397e143e41\") " pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.659374 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659298    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af2b6355-1e09-42dd-b2b0-df9287eb8be5-tmp-dir\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.659374 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjmql\" (UniqueName: \"kubernetes.io/projected/af2b6355-1e09-42dd-b2b0-df9287eb8be5-kube-api-access-qjmql\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.659505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.659505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659406    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb69f92-4fac-4919-a5b1-023d4cf821f7-data-volume\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.659505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659433    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.659656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-certificates\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ea1d47b-fbc5-49b3-86d3-26397e143e41-cert\") pod \"ingress-canary-l5d5t\" (UID: \"1ea1d47b-fbc5-49b3-86d3-26397e143e41\") " pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.659656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.659772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-image-registry-private-configuration\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-trusted-ca\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb69f92-4fac-4919-a5b1-023d4cf821f7-crio-socket\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.659772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659727    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/af2b6355-1e09-42dd-b2b0-df9287eb8be5-config-volume\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.659772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2cj7\" (UniqueName: \"kubernetes.io/projected/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-api-access-r2cj7\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.659940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659801    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-ca-trust-extracted\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-bound-sa-token\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.659940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.659871    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vckf2\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-kube-api-access-vckf2\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.707246 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.707159    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:25.761148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb69f92-4fac-4919-a5b1-023d4cf821f7-data-volume\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.761148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-certificates\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ea1d47b-fbc5-49b3-86d3-26397e143e41-cert\") pod \"ingress-canary-l5d5t\" (UID: \"1ea1d47b-fbc5-49b3-86d3-26397e143e41\") " pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-image-registry-private-configuration\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-trusted-ca\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb69f92-4fac-4919-a5b1-023d4cf821f7-crio-socket\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/af2b6355-1e09-42dd-b2b0-df9287eb8be5-config-volume\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761347    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 13:18:25.761375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2cj7\" (UniqueName: \"kubernetes.io/projected/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-api-access-r2cj7\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761433    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:26.26141149 +0000 UTC m=+36.637362287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : configmap "kube-rbac-proxy" not found
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-ca-trust-extracted\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761472    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761487    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9b69dbf-p46zn: secret "image-registry-tls" not found
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-bound-sa-token\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckf2\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-kube-api-access-vckf2\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb69f92-4fac-4919-a5b1-023d4cf821f7-data-volume\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761554    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls podName:24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:26.261516422 +0000 UTC m=+36.637467180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls") pod "image-registry-7f9b69dbf-p46zn" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3") : secret "image-registry-tls" not found
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761614    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761621    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-installation-pull-secrets\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.761656 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761650    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:26.261640153 +0000 UTC m=+36.637590910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4vl\" (UniqueName: \"kubernetes.io/projected/1ea1d47b-fbc5-49b3-86d3-26397e143e41-kube-api-access-nz4vl\") pod \"ingress-canary-l5d5t\" (UID: \"1ea1d47b-fbc5-49b3-86d3-26397e143e41\") " pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af2b6355-1e09-42dd-b2b0-df9287eb8be5-tmp-dir\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjmql\" (UniqueName: \"kubernetes.io/projected/af2b6355-1e09-42dd-b2b0-df9287eb8be5-kube-api-access-qjmql\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761845    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:25.761873    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls podName:af2b6355-1e09-42dd-b2b0-df9287eb8be5 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:26.261864253 +0000 UTC m=+36.637815010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls") pod "dns-default-mv4jn" (UID: "af2b6355-1e09-42dd-b2b0-df9287eb8be5") : secret "dns-default-metrics-tls" not found
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761883    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-certificates\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.762021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.761947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-ca-trust-extracted\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.762391 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.762335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb69f92-4fac-4919-a5b1-023d4cf821f7-crio-socket\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.762835 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.762614    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af2b6355-1e09-42dd-b2b0-df9287eb8be5-tmp-dir\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.762835 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.762716    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-trusted-ca\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.762835 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.762760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/af2b6355-1e09-42dd-b2b0-df9287eb8be5-config-volume\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.766303 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.766276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-installation-pull-secrets\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.766765 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.766745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ea1d47b-fbc5-49b3-86d3-26397e143e41-cert\") pod \"ingress-canary-l5d5t\" (UID: \"1ea1d47b-fbc5-49b3-86d3-26397e143e41\") " pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.768592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.768570    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-image-registry-private-configuration\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.772679 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.772650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjmql\" (UniqueName: \"kubernetes.io/projected/af2b6355-1e09-42dd-b2b0-df9287eb8be5-kube-api-access-qjmql\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:25.773006 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.772988    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-bound-sa-token\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.773787 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.773742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2cj7\" (UniqueName: \"kubernetes.io/projected/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-api-access-r2cj7\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:25.774274 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.774251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4vl\" (UniqueName: \"kubernetes.io/projected/1ea1d47b-fbc5-49b3-86d3-26397e143e41-kube-api-access-nz4vl\") pod \"ingress-canary-l5d5t\" (UID: \"1ea1d47b-fbc5-49b3-86d3-26397e143e41\") " pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.775336 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.775314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vckf2\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-kube-api-access-vckf2\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:25.837273 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.837236    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l5d5t"
Jul 23 13:18:25.927272 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.927239    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-glfww"]
Jul 23 13:18:25.930021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.929992    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:25.933230 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.933080    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 23 13:18:25.933230 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.933162    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 23 13:18:25.933689 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.933574    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 23 13:18:25.933875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.933858    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 23 13:18:25.934071 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.934056    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mvm54\""
Jul 23 13:18:25.939291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.939228    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-glfww"]
Jul 23 13:18:25.972937 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.972873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:25.996749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:25.996727    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l5d5t"]
Jul 23 13:18:26.001149 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:26.001125    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea1d47b_fbc5_49b3_86d3_26397e143e41.slice/crio-d283359116f5fe2a2661c91fa73827323e508943eddbba2f8af96698cf41b383 WatchSource:0}: Error finding container d283359116f5fe2a2661c91fa73827323e508943eddbba2f8af96698cf41b383: Status 404 returned error can't find the container with id d283359116f5fe2a2661c91fa73827323e508943eddbba2f8af96698cf41b383
Jul 23 13:18:26.063789 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.063765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8rpm\" (UniqueName: \"kubernetes.io/projected/00a88f2c-20df-49e2-8074-a343021e1f8b-kube-api-access-c8rpm\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.063908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.063817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/00a88f2c-20df-49e2-8074-a343021e1f8b-signing-key\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.063908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.063895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/00a88f2c-20df-49e2-8074-a343021e1f8b-signing-cabundle\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.075146 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.075108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:26.164602 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.164575    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/00a88f2c-20df-49e2-8074-a343021e1f8b-signing-cabundle\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.165481 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.164685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8rpm\" (UniqueName: \"kubernetes.io/projected/00a88f2c-20df-49e2-8074-a343021e1f8b-kube-api-access-c8rpm\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.165481 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.164724    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/00a88f2c-20df-49e2-8074-a343021e1f8b-signing-key\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.165481 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.165384    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/00a88f2c-20df-49e2-8074-a343021e1f8b-signing-cabundle\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.167736 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.167713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/00a88f2c-20df-49e2-8074-a343021e1f8b-signing-key\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.173566 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.173527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:26.174155 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.174136    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8rpm\" (UniqueName: \"kubernetes.io/projected/00a88f2c-20df-49e2-8074-a343021e1f8b-kube-api-access-c8rpm\") pod \"service-ca-58598695f7-glfww\" (UID: \"00a88f2c-20df-49e2-8074-a343021e1f8b\") " pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.243182 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.243112    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-glfww"
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.265990    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.266062    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.266129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266146    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.266165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266200    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266214    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:27.26619726 +0000 UTC m=+37.642148028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266219    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9b69dbf-p46zn: secret "image-registry-tls" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266250    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266269    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls podName:24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:27.266250618 +0000 UTC m=+37.642201373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls") pod "image-registry-7f9b69dbf-p46zn" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3") : secret "image-registry-tls" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266288    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:27.266277301 +0000 UTC m=+37.642228058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : configmap "kube-rbac-proxy" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266321    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 13:18:26.266014 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:26.266354    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls podName:af2b6355-1e09-42dd-b2b0-df9287eb8be5 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:27.266343924 +0000 UTC m=+37.642294681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls") pod "dns-default-mv4jn" (UID: "af2b6355-1e09-42dd-b2b0-df9287eb8be5") : secret "dns-default-metrics-tls" not found
Jul 23 13:18:26.274901 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.274876    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:26.308983 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.308755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l5d5t" event={"ID":"1ea1d47b-fbc5-49b3-86d3-26397e143e41","Type":"ContainerStarted","Data":"d283359116f5fe2a2661c91fa73827323e508943eddbba2f8af96698cf41b383"}
Jul 23 13:18:26.382764 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:26.382735    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-glfww"]
Jul 23 13:18:26.385923 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:26.385887    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a88f2c_20df_49e2_8074_a343021e1f8b.slice/crio-4c970f280d0c0de328ade7fab66d480facdf399b507687f43932500125230fe7 WatchSource:0}: Error finding container 4c970f280d0c0de328ade7fab66d480facdf399b507687f43932500125230fe7: Status 404 returned error can't find the container with id 4c970f280d0c0de328ade7fab66d480facdf399b507687f43932500125230fe7
Jul 23 13:18:27.085939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.085904    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:27.085939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.085924    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:27.086164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.085904    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:27.092079 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.092049    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 13:18:27.093426 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.093021    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 13:18:27.093426 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.093038    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 13:18:27.093426 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.093132    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-42dd2\""
Jul 23 13:18:27.093426 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.093025    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 13:18:27.093426 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.093286    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4ww68\""
Jul 23 13:18:27.109120 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.109093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:27.274848 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.274815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.274866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.274916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.274963    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275032    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275061    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls podName:af2b6355-1e09-42dd-b2b0-df9287eb8be5 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:29.275041074 +0000 UTC m=+39.650991847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls") pod "dns-default-mv4jn" (UID: "af2b6355-1e09-42dd-b2b0-df9287eb8be5") : secret "dns-default-metrics-tls" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275088    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:29.275077341 +0000 UTC m=+39.651028102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275098    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275132    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:29.275121332 +0000 UTC m=+39.651072088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : configmap "kube-rbac-proxy" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.275031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275224    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275232    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9b69dbf-p46zn: secret "image-registry-tls" not found
Jul 23 13:18:27.275288 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:27.275254    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls podName:24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:29.275247423 +0000 UTC m=+39.651198179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls") pod "image-registry-7f9b69dbf-p46zn" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3") : secret "image-registry-tls" not found
Jul 23 13:18:27.312812 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:27.312745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-glfww" event={"ID":"00a88f2c-20df-49e2-8074-a343021e1f8b","Type":"ContainerStarted","Data":"4c970f280d0c0de328ade7fab66d480facdf399b507687f43932500125230fe7"}
Jul 23 13:18:28.651249 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:28.651186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:29.019502 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:29.019439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:29.294591 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:29.294506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:29.294740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:29.294610    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:29.294740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:29.294646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:29.294740 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294656    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 13:18:29.294740 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294679    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9b69dbf-p46zn: secret "image-registry-tls" not found
Jul 23 13:18:29.294740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:29.294691    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:29.294740 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294742    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls podName:24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:33.294721585 +0000 UTC m=+43.670672341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls") pod "image-registry-7f9b69dbf-p46zn" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3") : secret "image-registry-tls" not found
Jul 23 13:18:29.295061 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294792    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:29.295061 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294795    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 13:18:29.295061 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294849    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:33.294831987 +0000 UTC m=+43.670782742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:29.295061 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:29.294873    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls podName:af2b6355-1e09-42dd-b2b0-df9287eb8be5 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:33.294856444 +0000 UTC m=+43.670807203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls") pod "dns-default-mv4jn" (UID: "af2b6355-1e09-42dd-b2b0-df9287eb8be5") : secret "dns-default-metrics-tls" not found
Jul 23 13:18:29.295216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:29.295196    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb69f92-4fac-4919-a5b1-023d4cf821f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:30.263446 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:30.263411    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:30.321966 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:30.321939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l5d5t" event={"ID":"1ea1d47b-fbc5-49b3-86d3-26397e143e41","Type":"ContainerStarted","Data":"5c126f65091309f052727ea1f1bfdf4d92c3555642eb1810becc53b3113d1bb9"}
Jul 23 13:18:30.340999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:30.340949    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l5d5t" podStartSLOduration=1.158957972 podStartE2EDuration="5.340930471s" totalImagesPullingTime="4.181972499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:30.34076484 +0000 UTC m=+40.716715619" watchObservedRunningTime="2026-07-23 13:18:30.340930471 +0000 UTC m=+40.716881228"
Jul 23 13:18:31.328730 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:31.328698    2559 generic.go:356] "Generic (PLEG): container finished" podID="c93e2ef1-8d3a-4194-ac04-7643f32891a4" containerID="18b964bb9b333c1d00c448dd6df488a02fe36ba98ddc34de8e8bed73fdd78dad" exitCode=0
Jul 23 13:18:31.329134 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:31.328763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerDied","Data":"18b964bb9b333c1d00c448dd6df488a02fe36ba98ddc34de8e8bed73fdd78dad"}
Jul 23 13:18:31.330405 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:31.330376    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-glfww" event={"ID":"00a88f2c-20df-49e2-8074-a343021e1f8b","Type":"ContainerStarted","Data":"f7bf05ea74f043a6bea17df7f96e2687f6c7dac395cfa6c8d8d829a30a7ef12c"}
Jul 23 13:18:31.386303 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:31.386241    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-glfww" podStartSLOduration=2.32236414 podStartE2EDuration="6.386225621s" totalImagesPullingTime="4.063861481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:31.385678119 +0000 UTC m=+41.761628898" watchObservedRunningTime="2026-07-23 13:18:31.386225621 +0000 UTC m=+41.762176400"
Jul 23 13:18:31.520437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:31.520396    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:31.805791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:31.805701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:32.336616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:32.336580    2559 generic.go:356] "Generic (PLEG): container finished" podID="c93e2ef1-8d3a-4194-ac04-7643f32891a4" containerID="8188b60754d2f540e293ec603f89776615b4d80b5fad67da51581d56116e78d0" exitCode=0
Jul 23 13:18:32.337044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:32.336655    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerDied","Data":"8188b60754d2f540e293ec603f89776615b4d80b5fad67da51581d56116e78d0"}
Jul 23 13:18:33.249201 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.249169    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:33.322311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.322286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:33.326260 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.326241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:33.326338 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.326288    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:33.326382 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.326343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:33.326422 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326385    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 13:18:33.326463 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326439    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 13:18:33.326463 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326451    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9b69dbf-p46zn: secret "image-registry-tls" not found
Jul 23 13:18:33.326463 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326455    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:33.326585 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326440    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls podName:af2b6355-1e09-42dd-b2b0-df9287eb8be5 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:41.326423129 +0000 UTC m=+51.702373887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls") pod "dns-default-mv4jn" (UID: "af2b6355-1e09-42dd-b2b0-df9287eb8be5") : secret "dns-default-metrics-tls" not found
Jul 23 13:18:33.326585 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326490    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls podName:24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:41.326478097 +0000 UTC m=+51.702428853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls") pod "image-registry-7f9b69dbf-p46zn" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3") : secret "image-registry-tls" not found
Jul 23 13:18:33.326585 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:18:33.326504    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls podName:acb69f92-4fac-4919-a5b1-023d4cf821f7 nodeName:}" failed. No retries permitted until 2026-07-23 13:18:41.326496079 +0000 UTC m=+51.702446834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmxhx" (UID: "acb69f92-4fac-4919-a5b1-023d4cf821f7") : secret "insights-runtime-extractor-tls" not found
Jul 23 13:18:33.342103 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.342076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" event={"ID":"c93e2ef1-8d3a-4194-ac04-7643f32891a4","Type":"ContainerStarted","Data":"2ee152e6442755d63d61eca03373bad411b4b90b0cdfacb350c63cac07a76573"}
Jul 23 13:18:33.380164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.380078    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6ltzr" podStartSLOduration=10.809793886 podStartE2EDuration="44.380055036s" totalImagesPullingTime="33.57026115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:33.377670802 +0000 UTC m=+43.753621580" watchObservedRunningTime="2026-07-23 13:18:33.380055036 +0000 UTC m=+43.756005810"
Jul 23 13:18:33.802831 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:33.802796    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:34.202941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:34.202908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:34.395542 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:34.395497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:34.589407 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:34.589365    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:34.991979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:34.991890    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:35.187652 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:35.187622    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:35.245254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:35.245184    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:35.801751 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:35.801721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:35.902432 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:35.902390    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:36.033177 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:36.033140    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:36.096373 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:36.096348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:36.409746 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:36.409666    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:36.592796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:36.592764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:36.656790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:36.656763    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:36.982030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:36.981999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:37.189305 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:37.189276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:37.582213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:37.582179    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:38.187583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:38.187552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:38.786129 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:38.786087    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:40.272744 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:40.272710    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:40.372930 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:40.372898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:40.785899 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:40.785873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:40.991284 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:40.991254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:41.187938 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.187909    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:41.384912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.384882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:41.385440 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.384932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:41.385440 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.384972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:41.388178 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.388147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb69f92-4fac-4919-a5b1-023d4cf821f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmxhx\" (UID: \"acb69f92-4fac-4919-a5b1-023d4cf821f7\") " pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:41.388299 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.388186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af2b6355-1e09-42dd-b2b0-df9287eb8be5-metrics-tls\") pod \"dns-default-mv4jn\" (UID: \"af2b6355-1e09-42dd-b2b0-df9287eb8be5\") " pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:41.388299 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.388244    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"image-registry-7f9b69dbf-p46zn\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") " pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:41.401039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.401021    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:41.409147 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.409132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmxhx"
Jul 23 13:18:41.417328 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.417314    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:41.428862 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.428833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:41.603658 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.603506    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f9b69dbf-p46zn"]
Jul 23 13:18:41.606097 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:41.606070    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24eb3f8e_c0f4_4063_b54d_2f3ec8ac28d3.slice/crio-5f1c89af5f036d806edfe6ceeb0db9df3bc1334c056f67393ee05989c29521da WatchSource:0}: Error finding container 5f1c89af5f036d806edfe6ceeb0db9df3bc1334c056f67393ee05989c29521da: Status 404 returned error can't find the container with id 5f1c89af5f036d806edfe6ceeb0db9df3bc1334c056f67393ee05989c29521da
Jul 23 13:18:41.611917 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.611893    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmxhx"]
Jul 23 13:18:41.615144 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:41.615121    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb69f92_4fac_4919_a5b1_023d4cf821f7.slice/crio-97c925413f3bd71d9ebdb09c3883387a0a4addab9164c03907a3eae19264cd08 WatchSource:0}: Error finding container 97c925413f3bd71d9ebdb09c3883387a0a4addab9164c03907a3eae19264cd08: Status 404 returned error can't find the container with id 97c925413f3bd71d9ebdb09c3883387a0a4addab9164c03907a3eae19264cd08
Jul 23 13:18:41.622197 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:41.622172    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mv4jn"]
Jul 23 13:18:41.625991 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:41.625971    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf2b6355_1e09_42dd_b2b0_df9287eb8be5.slice/crio-5717b298aa12884a6f04de2cda97ff81f7a56139328f822ecc22f5caba02d536 WatchSource:0}: Error finding container 5717b298aa12884a6f04de2cda97ff81f7a56139328f822ecc22f5caba02d536: Status 404 returned error can't find the container with id 5717b298aa12884a6f04de2cda97ff81f7a56139328f822ecc22f5caba02d536
Jul 23 13:18:42.004505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.004424    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:42.191514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.191466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:42.367078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.366982    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmxhx" event={"ID":"acb69f92-4fac-4919-a5b1-023d4cf821f7","Type":"ContainerStarted","Data":"97c925413f3bd71d9ebdb09c3883387a0a4addab9164c03907a3eae19264cd08"}
Jul 23 13:18:42.368737 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.368704    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mv4jn" event={"ID":"af2b6355-1e09-42dd-b2b0-df9287eb8be5","Type":"ContainerStarted","Data":"5717b298aa12884a6f04de2cda97ff81f7a56139328f822ecc22f5caba02d536"}
Jul 23 13:18:42.371752 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.371725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" event={"ID":"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3","Type":"ContainerStarted","Data":"83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c"}
Jul 23 13:18:42.371884 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.371773    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" event={"ID":"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3","Type":"ContainerStarted","Data":"5f1c89af5f036d806edfe6ceeb0db9df3bc1334c056f67393ee05989c29521da"}
Jul 23 13:18:42.372566 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.372512    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:18:42.393269 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:42.392185    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" podStartSLOduration=33.392166195 podStartE2EDuration="33.392166195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:42.391806733 +0000 UTC m=+52.767757514" watchObservedRunningTime="2026-07-23 13:18:42.392166195 +0000 UTC m=+52.768116967"
Jul 23 13:18:47.211166 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.211077    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f9b69dbf-p46zn"]
Jul 23 13:18:47.212240 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.212212    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"]
Jul 23 13:18:47.213979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.213962    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"]
Jul 23 13:18:47.214114 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.214096    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.215655 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.215633    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.216060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.216043    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"]
Jul 23 13:18:47.216856 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.216804    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 23 13:18:47.216989 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.216860    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 23 13:18:47.217269 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.217250    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 23 13:18:47.217352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.217293    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 23 13:18:47.217806 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.217790    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.217908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.217892    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-s482v\""
Jul 23 13:18:47.218273 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.218258    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 23 13:18:47.220160 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.220137    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 23 13:18:47.220160 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.220158    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 23 13:18:47.220401 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.220383    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 23 13:18:47.220619 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.220601    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 23 13:18:47.234466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.234439    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"]
Jul 23 13:18:47.235077 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.235060    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"]
Jul 23 13:18:47.238204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.238182    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"]
Jul 23 13:18:47.298352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.298321    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85bc644f4f-s5h7z"]
Jul 23 13:18:47.301271 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.301250    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.323998 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.323977    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85bc644f4f-s5h7z"]
Jul 23 13:18:47.340872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.340851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgtrz\" (UniqueName: \"kubernetes.io/projected/5faa3be4-190c-4062-8863-93d0063e1d20-kube-api-access-sgtrz\") pod \"managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7\" (UID: \"5faa3be4-190c-4062-8863-93d0063e1d20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.340976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.340887    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5faa3be4-190c-4062-8863-93d0063e1d20-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7\" (UID: \"5faa3be4-190c-4062-8863-93d0063e1d20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.340976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.340911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-registry-certificates\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341085 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.340969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.341085 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-trusted-ca\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341085 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341050    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-ca-trust-extracted\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-bound-sa-token\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.341236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-installation-pull-secrets\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-hub\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.341236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341191    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-registry-tls\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341216    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvx84\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-kube-api-access-wvx84\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341434 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vhc9\" (UniqueName: \"kubernetes.io/projected/e089226c-e187-4fac-bc2a-a78de0812f92-kube-api-access-6vhc9\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.341434 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.341434 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341309    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e089226c-e187-4fac-bc2a-a78de0812f92-klusterlet-config\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.341434 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341359    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-ca\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.341434 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-image-registry-private-configuration\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.341434 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341428    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e089226c-e187-4fac-bc2a-a78de0812f92-tmp\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.341648 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.341444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slkw4\" (UniqueName: \"kubernetes.io/projected/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-kube-api-access-slkw4\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.381982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.381954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:47.392838 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.392810    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mv4jn" event={"ID":"af2b6355-1e09-42dd-b2b0-df9287eb8be5","Type":"ContainerStarted","Data":"adcf88c6919e661804a597170154e14de6bb98641308fc7990ab454cddc2b889"}
Jul 23 13:18:47.392971 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.392893    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mv4jn" event={"ID":"af2b6355-1e09-42dd-b2b0-df9287eb8be5","Type":"ContainerStarted","Data":"cb52c8a22b52d3aba741e7e5ac899ff0be7eb0113c51e5db797335e1955a2378"}
Jul 23 13:18:47.392971 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.392923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:47.394221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.394200    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmxhx" event={"ID":"acb69f92-4fac-4919-a5b1-023d4cf821f7","Type":"ContainerStarted","Data":"3fe8e65ebee7820f6c80eeb2619f83a1c7870baba55b068e0294c021a49c6663"}
Jul 23 13:18:47.416906 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.416869    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mv4jn" podStartSLOduration=17.215493933 podStartE2EDuration="22.416858125s" totalImagesPullingTime="5.201364192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:47.416138492 +0000 UTC m=+57.792089270" watchObservedRunningTime="2026-07-23 13:18:47.416858125 +0000 UTC m=+57.792808896"
Jul 23 13:18:47.442144 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442125    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vhc9\" (UniqueName: \"kubernetes.io/projected/e089226c-e187-4fac-bc2a-a78de0812f92-kube-api-access-6vhc9\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.442256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.442256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e089226c-e187-4fac-bc2a-a78de0812f92-klusterlet-config\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.442256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442200    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-ca\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.442405 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442340    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-image-registry-private-configuration\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.442526 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442505    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e089226c-e187-4fac-bc2a-a78de0812f92-tmp\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.442673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slkw4\" (UniqueName: \"kubernetes.io/projected/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-kube-api-access-slkw4\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.442814 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtrz\" (UniqueName: \"kubernetes.io/projected/5faa3be4-190c-4062-8863-93d0063e1d20-kube-api-access-sgtrz\") pod \"managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7\" (UID: \"5faa3be4-190c-4062-8863-93d0063e1d20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.442974 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442930    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e089226c-e187-4fac-bc2a-a78de0812f92-tmp\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.442974 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5faa3be4-190c-4062-8863-93d0063e1d20-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7\" (UID: \"5faa3be4-190c-4062-8863-93d0063e1d20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.442974 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.442990    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-registry-certificates\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.442974 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.443310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-trusted-ca\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.443310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-ca-trust-extracted\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.443310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-bound-sa-token\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.443310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.443310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443246    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-installation-pull-secrets\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.443615 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.443592    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-hub\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.444299 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.444273    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-trusted-ca\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.444391 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.444334    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-registry-tls\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.444391 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.444369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvx84\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-kube-api-access-wvx84\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.444680 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.444656    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-registry-certificates\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.444970 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.444931    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.445330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.445310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-ca-trust-extracted\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.445713 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.445578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.445892 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.445740    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e089226c-e187-4fac-bc2a-a78de0812f92-klusterlet-config\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.447412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.446406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-image-registry-private-configuration\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.448060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.447943    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-registry-tls\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.448060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.447995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-installation-pull-secrets\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.448060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.448039    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-hub\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.448060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.448053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-ca\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.448320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.448097    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.448374 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.448332    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5faa3be4-190c-4062-8863-93d0063e1d20-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7\" (UID: \"5faa3be4-190c-4062-8863-93d0063e1d20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.465634 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.465582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vhc9\" (UniqueName: \"kubernetes.io/projected/e089226c-e187-4fac-bc2a-a78de0812f92-kube-api-access-6vhc9\") pod \"klusterlet-addon-workmgr-7db64d4787-mfl5k\" (UID: \"e089226c-e187-4fac-bc2a-a78de0812f92\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.472632 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.472608    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slkw4\" (UniqueName: \"kubernetes.io/projected/ac4138d0-fa05-4e68-bb31-fd7d528c1d9f-kube-api-access-slkw4\") pod \"cluster-proxy-proxy-agent-99d5d668b-ccll4\" (UID: \"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.472814 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.472798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtrz\" (UniqueName: \"kubernetes.io/projected/5faa3be4-190c-4062-8863-93d0063e1d20-kube-api-access-sgtrz\") pod \"managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7\" (UID: \"5faa3be4-190c-4062-8863-93d0063e1d20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.473875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.473857    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvx84\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-kube-api-access-wvx84\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.475854 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.475838    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34b9c9e6-7618-4c40-a6a0-ec40f5551d52-bound-sa-token\") pod \"image-registry-85bc644f4f-s5h7z\" (UID: \"34b9c9e6-7618-4c40-a6a0-ec40f5551d52\") " pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.526033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.526010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:47.541052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.541022    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"
Jul 23 13:18:47.546081 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.546061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"
Jul 23 13:18:47.625208 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.624887    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:47.694445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.694410    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"]
Jul 23 13:18:47.697763 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:47.697734    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode089226c_e187_4fac_bc2a_a78de0812f92.slice/crio-c1b5897bca1040374cb66bd8da263f59f146fbf9c199eb15aa2ac975b90433a8 WatchSource:0}: Error finding container c1b5897bca1040374cb66bd8da263f59f146fbf9c199eb15aa2ac975b90433a8: Status 404 returned error can't find the container with id c1b5897bca1040374cb66bd8da263f59f146fbf9c199eb15aa2ac975b90433a8
Jul 23 13:18:47.778796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.778774    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85bc644f4f-s5h7z"]
Jul 23 13:18:47.780816 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:47.780791    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b9c9e6_7618_4c40_a6a0_ec40f5551d52.slice/crio-4e6a6dbea83c801660a74e357fc70583c4f5760f7ddd42ba336953ce39b08fc2 WatchSource:0}: Error finding container 4e6a6dbea83c801660a74e357fc70583c4f5760f7ddd42ba336953ce39b08fc2: Status 404 returned error can't find the container with id 4e6a6dbea83c801660a74e357fc70583c4f5760f7ddd42ba336953ce39b08fc2
Jul 23 13:18:47.787660 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.787636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:47.898291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.898251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:47.910267 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.910244    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7"]
Jul 23 13:18:47.914184 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:18:47.914158    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5faa3be4_190c_4062_8863_93d0063e1d20.slice/crio-466b2f3f6a06dede591c56a37f405f99455a231f806d89bbc9beb2bc39f5115c WatchSource:0}: Error finding container 466b2f3f6a06dede591c56a37f405f99455a231f806d89bbc9beb2bc39f5115c: Status 404 returned error can't find the container with id 466b2f3f6a06dede591c56a37f405f99455a231f806d89bbc9beb2bc39f5115c
Jul 23 13:18:47.917201 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:47.917178    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4"]
Jul 23 13:18:48.247506 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.247476    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:48.347731 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.347701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:48.399236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.399208    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z" event={"ID":"34b9c9e6-7618-4c40-a6a0-ec40f5551d52","Type":"ContainerStarted","Data":"2edd8e0441dc62f5002ea31106bb7b8ca872c76b985d5bd82be3bc4d25057fb7"}
Jul 23 13:18:48.399449 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.399262    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z" event={"ID":"34b9c9e6-7618-4c40-a6a0-ec40f5551d52","Type":"ContainerStarted","Data":"4e6a6dbea83c801660a74e357fc70583c4f5760f7ddd42ba336953ce39b08fc2"}
Jul 23 13:18:48.399449 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.399291    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:18:48.400423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.400399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4" event={"ID":"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f","Type":"ContainerStarted","Data":"d4cc04a84ec10b000c47b1ce8d842817b85d268c176c90b6895dd141c706b962"}
Jul 23 13:18:48.401502 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.401480    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7" event={"ID":"5faa3be4-190c-4062-8863-93d0063e1d20","Type":"ContainerStarted","Data":"466b2f3f6a06dede591c56a37f405f99455a231f806d89bbc9beb2bc39f5115c"}
Jul 23 13:18:48.402572 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.402549    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k" event={"ID":"e089226c-e187-4fac-bc2a-a78de0812f92","Type":"ContainerStarted","Data":"c1b5897bca1040374cb66bd8da263f59f146fbf9c199eb15aa2ac975b90433a8"}
Jul 23 13:18:48.419165 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.419126    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z" podStartSLOduration=1.4191122090000001 podStartE2EDuration="1.419112209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:48.418745182 +0000 UTC m=+58.794695961" watchObservedRunningTime="2026-07-23 13:18:48.419112209 +0000 UTC m=+58.795063009"
Jul 23 13:18:48.645595 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.645565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:48.750805 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:48.750769    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:49.452403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:49.452349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:49.552666 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:49.552507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:49.984089 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:49.984048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:50.151788 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:50.151754    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:50.940843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:50.940806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:50.999435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:50.999348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:51.308983 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:51.308917    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qbvpc"
Jul 23 13:18:51.563058 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:51.562976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:51.664472 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:51.664436    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:52.532216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:52.532174    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:52.636015 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:52.635980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:52.694473 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:52.694439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:52.811214 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:52.811118    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:52.985980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:52.985950    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:53.062953 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:53.062867    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:53.185256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:53.185214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:53.415089 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:53.415048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:53.520203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:53.520165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:53.723321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:53.723242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:53.804308 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:53.804270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:54.013729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:54.013652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:54.360871 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:54.360843    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:54.458131 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:54.458094    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:54.545487 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:54.545459    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:54.613981 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:54.613890    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:54.747393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:54.747359    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:55.007243 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.007148    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:55.824463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.824416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:55.824933 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.824482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:55.827370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.827347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 13:18:55.827499 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.827349    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 13:18:55.837286 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.837261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f498563d-5c40-4fdb-b1b1-6b255440eff6-original-pull-secret\") pod \"global-pull-secret-syncer-z5rhr\" (UID: \"f498563d-5c40-4fdb-b1b1-6b255440eff6\") " pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:55.837401 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.837340    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4468347-7f97-4755-8ed3-44c90df023d3-metrics-certs\") pod \"network-metrics-daemon-dn6zg\" (UID: \"e4468347-7f97-4755-8ed3-44c90df023d3\") " pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:55.908259 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.908218    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z5rhr"
Jul 23 13:18:55.917722 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.917695    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-42dd2\""
Jul 23 13:18:55.925193 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:55.925157    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dn6zg"
Jul 23 13:18:56.026411 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.026370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:56.029686 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.029657    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 13:18:56.040075 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.040040    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 13:18:56.050286 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.050252    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjlx6\" (UniqueName: \"kubernetes.io/projected/bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481-kube-api-access-kjlx6\") pod \"network-check-target-6bf7k\" (UID: \"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481\") " pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:56.202893 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.202861    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4ww68\""
Jul 23 13:18:56.210205 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.210180    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:18:56.588433 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.588336    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dn6zg"]
Jul 23 13:18:56.613018 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.612990    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z5rhr"]
Jul 23 13:18:56.614373 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:56.614025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6bf7k"]
Jul 23 13:18:57.217909 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.217870    2559 patch_prober.go:28] interesting pod/image-registry-7f9b69dbf-p46zn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 13:18:57.218363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.217946    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" podUID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 13:18:57.405889 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.405854    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mv4jn"
Jul 23 13:18:57.445735 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.445693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmxhx" event={"ID":"acb69f92-4fac-4919-a5b1-023d4cf821f7","Type":"ContainerStarted","Data":"467602ef3c368b6965edbbf77e60cefb0d27a38b72154323e598dabbb78c3eaf"}
Jul 23 13:18:57.448683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.448651    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4" event={"ID":"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f","Type":"ContainerStarted","Data":"53ddf851666c9530facff7c339969920a2ef59a4908b44f692981eea67623049"}
Jul 23 13:18:57.448823 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.448698    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4" event={"ID":"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f","Type":"ContainerStarted","Data":"fbf8ee422b63670862ac0a90f443cb980b91137d159eac61c2baf235509a5f7b"}
Jul 23 13:18:57.448823 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.448717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4" event={"ID":"ac4138d0-fa05-4e68-bb31-fd7d528c1d9f","Type":"ContainerStarted","Data":"975123db2301417f262d89b05c8cfda28cf50029e97666ff781b203cb5ec075c"}
Jul 23 13:18:57.451241 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.451195    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7" event={"ID":"5faa3be4-190c-4062-8863-93d0063e1d20","Type":"ContainerStarted","Data":"fd1d0a4da668a585a24458f995e62ab5e576ee85623c48fb02ed43b94b24d3fb"}
Jul 23 13:18:57.452740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.452716    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k" event={"ID":"e089226c-e187-4fac-bc2a-a78de0812f92","Type":"ContainerStarted","Data":"80ffc5ea62467475a9e1d92e62c5516a063776bc9a293a602b8fb3d48ad8c9e8"}
Jul 23 13:18:57.453215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.453170    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:57.454471 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.454138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z5rhr" event={"ID":"f498563d-5c40-4fdb-b1b1-6b255440eff6","Type":"ContainerStarted","Data":"691bedcc4f823b606e4a1953506d5e5bf0b04db0d7588ec44f888e187690a690"}
Jul 23 13:18:57.454922 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.454901    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k"
Jul 23 13:18:57.455877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.455854    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6bf7k" event={"ID":"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481","Type":"ContainerStarted","Data":"cdd40fb59c7c53bb48d6c3b50b0733121db8bc6772968bfaec30d2355f644744"}
Jul 23 13:18:57.457484 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.457466    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dn6zg" event={"ID":"e4468347-7f97-4755-8ed3-44c90df023d3","Type":"ContainerStarted","Data":"de0b60103572f77471e2100b6587a8e03eb28e2ba0af701eb35e14a23ea39e79"}
Jul 23 13:18:57.486123 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.485973    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99d5d668b-ccll4" podStartSLOduration=2.011258033 podStartE2EDuration="10.485951581s" totalImagesPullingTime="8.474693548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:57.46924379 +0000 UTC m=+67.845194570" watchObservedRunningTime="2026-07-23 13:18:57.485951581 +0000 UTC m=+67.861902351"
Jul 23 13:18:57.486307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:57.486272    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7db64d4787-mfl5k" podStartSLOduration=1.762144745 podStartE2EDuration="10.486259067s" totalImagesPullingTime="8.724114322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:57.484696272 +0000 UTC m=+67.860647050" watchObservedRunningTime="2026-07-23 13:18:57.486259067 +0000 UTC m=+67.862209847"
Jul 23 13:18:58.463105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:58.462519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dn6zg" event={"ID":"e4468347-7f97-4755-8ed3-44c90df023d3","Type":"ContainerStarted","Data":"d30ee08e4bac87adfbbc3b53c716bfebc1d6912799cb903422ba2809457cee10"}
Jul 23 13:18:58.829700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:58.829664    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:58.909136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:58.909099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:59.344646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:59.344612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:59.471033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:59.470991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dn6zg" event={"ID":"e4468347-7f97-4755-8ed3-44c90df023d3","Type":"ContainerStarted","Data":"64fafbcbd74616a3b87877392e4f32c9e6ccf8f931ba6f4d0af0b383fbcd6923"}
Jul 23 13:18:59.495243 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:59.494890    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-867c4fd7f6-2bjj7" podStartSLOduration=4.012225844 podStartE2EDuration="12.494866838s" totalImagesPullingTime="8.482640994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:57.50160991 +0000 UTC m=+67.877560689" watchObservedRunningTime="2026-07-23 13:18:59.494866838 +0000 UTC m=+69.870817616"
Jul 23 13:18:59.495243 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:59.495105    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dn6zg" podStartSLOduration=68.793606504 podStartE2EDuration="1m10.495096627s" totalImagesPullingTime="1.701490123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:59.492840902 +0000 UTC m=+69.868791679" watchObservedRunningTime="2026-07-23 13:18:59.495096627 +0000 UTC m=+69.871047406"
Jul 23 13:18:59.628036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:59.627902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:18:59.693856 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:18:59.693796    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:00.635993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:00.635966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:00.736962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:00.736922    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:01.255248 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:01.255206    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:01.324239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:01.324204    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:04.460312 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.460278    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:04.493591 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.493553    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z5rhr" event={"ID":"f498563d-5c40-4fdb-b1b1-6b255440eff6","Type":"ContainerStarted","Data":"76da6ef664cc89b3ee8a55a761c74b67866960969091c5a308777a6f0a129d68"}
Jul 23 13:19:04.495014 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.494989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6bf7k" event={"ID":"bafc0d0a-5294-4a3b-bba4-c1ddcdf7c481","Type":"ContainerStarted","Data":"a33fab6ba6589e3d138b3a5adf0036944fe8b3572b5c45e05119765741c84f4a"}
Jul 23 13:19:04.495146 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.495050    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:19:04.496962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.496943    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmxhx" event={"ID":"acb69f92-4fac-4919-a5b1-023d4cf821f7","Type":"ContainerStarted","Data":"c5eae5d0ee21ecbf89a4cd774af80c105da0d208edf6c85e9fe66f657c234fc5"}
Jul 23 13:19:04.509210 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.509169    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z5rhr" podStartSLOduration=68.421396992 podStartE2EDuration="1m15.509156541s" totalImagesPullingTime="7.087759549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:04.508302643 +0000 UTC m=+74.884253420" watchObservedRunningTime="2026-07-23 13:19:04.509156541 +0000 UTC m=+74.885107318"
Jul 23 13:19:04.519934 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.519886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:04.523878 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.523839    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6bf7k" podStartSLOduration=68.448630315 podStartE2EDuration="1m15.523825063s" totalImagesPullingTime="7.075194748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:04.522976783 +0000 UTC m=+74.898927566" watchObservedRunningTime="2026-07-23 13:19:04.523825063 +0000 UTC m=+74.899775841"
Jul 23 13:19:04.542169 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.542119    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cmxhx" podStartSLOduration=17.591389419 podStartE2EDuration="39.542100441s" totalImagesPullingTime="21.950711022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:18:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:04.541180286 +0000 UTC m=+74.917131064" watchObservedRunningTime="2026-07-23 13:19:04.542100441 +0000 UTC m=+74.918051218"
Jul 23 13:19:04.992820 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:04.992781    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.242247 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.242202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.293656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.293568    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f2tj4"]
Jul 23 13:19:05.296387 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.296366    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.299097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.299072    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 13:19:05.299219 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.299096    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 13:19:05.299219 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.299087    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 13:19:05.299508 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.299492    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 13:19:05.299732 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.299719    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 13:19:05.300344 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.300322    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v5kmg\""
Jul 23 13:19:05.300402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.300367    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 13:19:05.345237 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.345216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.409070 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409036    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-root\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-sys\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409121    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409163    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-tls\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409225    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-wtmp\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409249    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-metrics-client-ca\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409274    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-textfile\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.409402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.409296    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrbhv\" (UniqueName: \"kubernetes.io/projected/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-kube-api-access-wrbhv\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.435927 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.435898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.510610 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-root\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-sys\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-root\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-tls\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-wtmp\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510826    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-metrics-client-ca\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-textfile\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.510863    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrbhv\" (UniqueName: \"kubernetes.io/projected/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-kube-api-access-wrbhv\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511491 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:19:05.511152    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 13:19:05.511491 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:19:05.511194    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-tls podName:dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9 nodeName:}" failed. No retries permitted until 2026-07-23 13:19:06.011181261 +0000 UTC m=+76.387132017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-tls") pod "node-exporter-f2tj4" (UID: "dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9") : secret "node-exporter-tls" not found
Jul 23 13:19:05.511491 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.511221    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511491 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.511288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-sys\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511714 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.511512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-wtmp\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511714 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.511570    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-textfile\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.511919 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.511900    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-metrics-client-ca\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.513778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.513752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.520410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.520366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrbhv\" (UniqueName: \"kubernetes.io/projected/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-kube-api-access-wrbhv\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:05.536732 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.536701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.635444 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.635414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.736881 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.736850    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.852542 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.852512    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:05.951209 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:05.951137    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:06.016288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.016256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-tls\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:06.018768 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.018749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9-node-exporter-tls\") pod \"node-exporter-f2tj4\" (UID: \"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9\") " pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:06.208383 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.208300    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f2tj4"
Jul 23 13:19:06.220056 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:19:06.220025    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc46b3a1_80b6_4ba4_bea7_47cb0ca305c9.slice/crio-73764dc541493017b55f9fe772e3e5ef6dffa42c346ebef87aec7083f2234d27 WatchSource:0}: Error finding container 73764dc541493017b55f9fe772e3e5ef6dffa42c346ebef87aec7083f2234d27: Status 404 returned error can't find the container with id 73764dc541493017b55f9fe772e3e5ef6dffa42c346ebef87aec7083f2234d27
Jul 23 13:19:06.254673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.254642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:06.354701 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.354671    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:06.505859 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.505778    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f2tj4" event={"ID":"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9","Type":"ContainerStarted","Data":"73764dc541493017b55f9fe772e3e5ef6dffa42c346ebef87aec7083f2234d27"}
Jul 23 13:19:06.555024 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.554991    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:06.651264 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.651234    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:06.880293 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:06.880256    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:07.215635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:07.215559    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:19:07.234832 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:07.234805    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:07.334052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:07.334023    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:08.268239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:08.268203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:08.372291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:08.372258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:08.468976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:08.468899    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:08.568853 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:08.568748    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:09.411955 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.411929    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85bc644f4f-s5h7z"
Jul 23 13:19:09.518895 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.518861    2559 generic.go:356] "Generic (PLEG): container finished" podID="dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9" containerID="c61f1dc7187784be3befb1a111384ab1c08d49ecab96b94d77f971f62894fc1a" exitCode=0
Jul 23 13:19:09.519078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.518949    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f2tj4" event={"ID":"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9","Type":"ContainerDied","Data":"c61f1dc7187784be3befb1a111384ab1c08d49ecab96b94d77f971f62894fc1a"}
Jul 23 13:19:09.590794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.590763    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:09.659200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.659175    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:09.726316 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.726228    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:09.829941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:09.829907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:10.013796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.013701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:10.113346 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.113316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:10.220653 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.220620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:10.271098 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.271025    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f9fbf744d-h5m56"]
Jul 23 13:19:10.273989 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.273974    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.277801 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.277779    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 13:19:10.277912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.277800    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 13:19:10.277912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.277786    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 13:19:10.277912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.277892    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 13:19:10.279113 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.279095    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q4mmp\""
Jul 23 13:19:10.279200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.279182    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 13:19:10.279347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.279327    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 13:19:10.279466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.279363    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 13:19:10.283216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.283198    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 13:19:10.286849 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.286829    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f9fbf744d-h5m56"]
Jul 23 13:19:10.327198 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.327173    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:10.451351 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451325    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-oauth-serving-cert\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.451698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451361    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkmqm\" (UniqueName: \"kubernetes.io/projected/3505d1c8-f790-4e83-a417-a9e52f18a06d-kube-api-access-wkmqm\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.451698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451379    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-service-ca\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.451698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451426    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-config\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.451698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-trusted-ca-bundle\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.451698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-serving-cert\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.451698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.451486    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-oauth-config\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.476807 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.476781    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"]
Jul 23 13:19:10.480029 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.480011    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.483686 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.483664    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 23 13:19:10.483686 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.483686    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 23 13:19:10.484268 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.484248    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 23 13:19:10.484392 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.484261    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 23 13:19:10.484804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.484786    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 23 13:19:10.489458 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.489113    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-d25d8\""
Jul 23 13:19:10.492463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.492447    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 23 13:19:10.501012 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.500993    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"]
Jul 23 13:19:10.524721 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.524666    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f2tj4" event={"ID":"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9","Type":"ContainerStarted","Data":"70c8e76aae222dd278148f0df2ed3c2473abd5a00225362a1ed1c57f7f37abae"}
Jul 23 13:19:10.524797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.524717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f2tj4" event={"ID":"dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9","Type":"ContainerStarted","Data":"7c1a173896afba904eda6cd49842ed45442bbcaacd0d56462ddab970b911c02b"}
Jul 23 13:19:10.548945 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.548897    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f2tj4" podStartSLOduration=2.838097179 podStartE2EDuration="5.548883623s" totalImagesPullingTime="2.639996297s" totalInitContainerRuntime="70.790147ms" isStatefulPod=true podCreationTimestamp="2026-07-23 13:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:10.547853467 +0000 UTC m=+80.923804244" watchObservedRunningTime="2026-07-23 13:19:10.548883623 +0000 UTC m=+80.924834401"
Jul 23 13:19:10.552167 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkmqm\" (UniqueName: \"kubernetes.io/projected/3505d1c8-f790-4e83-a417-a9e52f18a06d-kube-api-access-wkmqm\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552222 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-service-ca\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-config\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-trusted-ca-bundle\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552389    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-serving-cert\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-oauth-config\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552599 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-oauth-serving-cert\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.552896 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.552866    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-service-ca\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.553051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.553032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-config\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.553377 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.553356    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-trusted-ca-bundle\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.553446 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.553391    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-oauth-serving-cert\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.554920 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.554893    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-serving-cert\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.554999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.554933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-oauth-config\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.562775 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.562757    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkmqm\" (UniqueName: \"kubernetes.io/projected/3505d1c8-f790-4e83-a417-a9e52f18a06d-kube-api-access-wkmqm\") pod \"console-7f9fbf744d-h5m56\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") " pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.583740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.583720    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:10.655222 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-federate-client-tls\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-telemeter-client-tls\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwswx\" (UniqueName: \"kubernetes.io/projected/593c31c9-7381-4b93-8137-b870e191b536-kube-api-access-fwswx\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655500 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655500 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-serving-certs-ca-bundle\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-secret-telemeter-client\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.655743 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.655667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-metrics-client-ca\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.721862 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.721840    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f9fbf744d-h5m56"]
Jul 23 13:19:10.723730 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:19:10.723698    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3505d1c8_f790_4e83_a417_a9e52f18a06d.slice/crio-d3a9d6d4faa1490d2276d436afcd85994d62be746a956388d26b55d46439806c WatchSource:0}: Error finding container d3a9d6d4faa1490d2276d436afcd85994d62be746a956388d26b55d46439806c: Status 404 returned error can't find the container with id d3a9d6d4faa1490d2276d436afcd85994d62be746a956388d26b55d46439806c
Jul 23 13:19:10.756201 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-secret-telemeter-client\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-metrics-client-ca\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-federate-client-tls\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-telemeter-client-tls\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwswx\" (UniqueName: \"kubernetes.io/projected/593c31c9-7381-4b93-8137-b870e191b536-kube-api-access-fwswx\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756324    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-serving-certs-ca-bundle\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.756588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.756388    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.757296 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.757229    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-metrics-client-ca\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.757416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.757386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-serving-certs-ca-bundle\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.757721 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.757695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/593c31c9-7381-4b93-8137-b870e191b536-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.758948 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.758918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-secret-telemeter-client\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.759097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.759079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-federate-client-tls\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.759097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.759086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.759171 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.759092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/593c31c9-7381-4b93-8137-b870e191b536-telemeter-client-tls\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.765135 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.765114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwswx\" (UniqueName: \"kubernetes.io/projected/593c31c9-7381-4b93-8137-b870e191b536-kube-api-access-fwswx\") pod \"telemeter-client-7f6dfcb74b-9jm4s\" (UID: \"593c31c9-7381-4b93-8137-b870e191b536\") " pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.791857 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.791807    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"
Jul 23 13:19:10.922401 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:10.922373    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s"]
Jul 23 13:19:10.925250 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:19:10.925226    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593c31c9_7381_4b93_8137_b870e191b536.slice/crio-be2ff9eb3a2cbf6ad41d1151ac9c34d4a1579a553dda5711ccc0700dc06bbb98 WatchSource:0}: Error finding container be2ff9eb3a2cbf6ad41d1151ac9c34d4a1579a553dda5711ccc0700dc06bbb98: Status 404 returned error can't find the container with id be2ff9eb3a2cbf6ad41d1151ac9c34d4a1579a553dda5711ccc0700dc06bbb98
Jul 23 13:19:11.024150 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:11.024113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:11.125954 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:11.125795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:11.511955 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:11.511848    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:11.530861 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:11.530824    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9fbf744d-h5m56" event={"ID":"3505d1c8-f790-4e83-a417-a9e52f18a06d","Type":"ContainerStarted","Data":"d3a9d6d4faa1490d2276d436afcd85994d62be746a956388d26b55d46439806c"}
Jul 23 13:19:11.532129 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:11.532101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s" event={"ID":"593c31c9-7381-4b93-8137-b870e191b536","Type":"ContainerStarted","Data":"be2ff9eb3a2cbf6ad41d1151ac9c34d4a1579a553dda5711ccc0700dc06bbb98"}
Jul 23 13:19:12.106322 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.106283    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:12.207864 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.207829    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:12.233437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.233393    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" podUID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" containerName="registry" containerID="cri-o://83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c" gracePeriod=30
Jul 23 13:19:12.505429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.505404    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:19:12.543051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.543013    2559 generic.go:356] "Generic (PLEG): container finished" podID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" containerID="83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c" exitCode=0
Jul 23 13:19:12.543410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.543063    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn"
Jul 23 13:19:12.543410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.543112    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" event={"ID":"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3","Type":"ContainerDied","Data":"83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c"}
Jul 23 13:19:12.543410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.543177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9b69dbf-p46zn" event={"ID":"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3","Type":"ContainerDied","Data":"5f1c89af5f036d806edfe6ceeb0db9df3bc1334c056f67393ee05989c29521da"}
Jul 23 13:19:12.543410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.543202    2559 scope.go:117] "RemoveContainer" containerID="83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c"
Jul 23 13:19:12.552330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.552310    2559 scope.go:117] "RemoveContainer" containerID="83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c"
Jul 23 13:19:12.552657 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:19:12.552629    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c\": container with ID starting with 83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c not found: ID does not exist" containerID="83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c"
Jul 23 13:19:12.552758 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.552667    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c"} err="failed to get container status \"83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c\": rpc error: code = NotFound desc = could not find container \"83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c\": container with ID starting with 83a2091d4e7cd94607c85c077a653509dc89016f2384d24e537fcfbe7c74f71c not found: ID does not exist"
Jul 23 13:19:12.672357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672279    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-installation-pull-secrets\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672315    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672348    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-trusted-ca\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672637 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672380    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-certificates\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672637 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672409    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-ca-trust-extracted\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672637 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672437    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-image-registry-private-configuration\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672637 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672467    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-bound-sa-token\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.672637 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.672489    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vckf2\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-kube-api-access-vckf2\") pod \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\" (UID: \"24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3\") "
Jul 23 13:19:12.674781 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.673274    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:19:12.674781 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.673311    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:19:12.676651 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.676089    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-kube-api-access-vckf2" (OuterVolumeSpecName: "kube-api-access-vckf2") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "kube-api-access-vckf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:19:12.676651 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.676526    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:19:12.678009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.677886    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:19:12.678108 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.678048    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:19:12.681819 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.681742    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:19:12.685385 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.685360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" (UID: "24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:19:12.773528 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773490    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-installation-pull-secrets\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773528 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773525    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-tls\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773528 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773552    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-trusted-ca\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773528 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773564    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-registry-certificates\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773575    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-ca-trust-extracted\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773585    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-image-registry-private-configuration\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773594    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-bound-sa-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.773875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.773602    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vckf2\" (UniqueName: \"kubernetes.io/projected/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3-kube-api-access-vckf2\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:12.867109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.867067    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f9b69dbf-p46zn"]
Jul 23 13:19:12.872550 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:12.872514    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f9b69dbf-p46zn"]
Jul 23 13:19:14.090960 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:14.090927    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" path="/var/lib/kubelet/pods/24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3/volumes"
Jul 23 13:19:14.554660 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:14.554569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9fbf744d-h5m56" event={"ID":"3505d1c8-f790-4e83-a417-a9e52f18a06d","Type":"ContainerStarted","Data":"7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5"}
Jul 23 13:19:14.556064 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:14.556039    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s" event={"ID":"593c31c9-7381-4b93-8137-b870e191b536","Type":"ContainerStarted","Data":"168789b3f5f493aab5b621835f1f233b661247abe19986fa702a82d648f28722"}
Jul 23 13:19:14.575166 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:14.575122    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f9fbf744d-h5m56" podStartSLOduration=1.129828164 podStartE2EDuration="4.575107378s" totalImagesPullingTime="3.445279214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:14.57320111 +0000 UTC m=+84.949151889" watchObservedRunningTime="2026-07-23 13:19:14.575107378 +0000 UTC m=+84.951058220"
Jul 23 13:19:15.553039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:15.552971    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:15.561598 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:15.561568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s" event={"ID":"593c31c9-7381-4b93-8137-b870e191b536","Type":"ContainerStarted","Data":"7f2bb94d5a15975112972bc2c80172c03a9d64c42fcfcb64502d871f66848e75"}
Jul 23 13:19:15.561729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:15.561632    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s" event={"ID":"593c31c9-7381-4b93-8137-b870e191b536","Type":"ContainerStarted","Data":"d0af3b1504db7db0e523568e42c63b922053e4245db45b730fcf4fb105f075ae"}
Jul 23 13:19:15.587587 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:15.587546    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f6dfcb74b-9jm4s" podStartSLOduration=1.234911975 podStartE2EDuration="5.587514656s" totalImagesPullingTime="4.352602681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:19:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:15.587142173 +0000 UTC m=+85.963092951" watchObservedRunningTime="2026-07-23 13:19:15.587514656 +0000 UTC m=+85.963465433"
Jul 23 13:19:15.654443 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:15.654364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:16.176113 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:16.176079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:16.244065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:16.244033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:16.746419 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:16.746333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:16.848700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:16.848666    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:17.125324 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.125289    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-794f8595d4-ssdgk"]
Jul 23 13:19:17.125644 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.125627    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" containerName="registry"
Jul 23 13:19:17.125743 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.125646    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" containerName="registry"
Jul 23 13:19:17.125743 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.125717    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="24eb3f8e-c0f4-4063-b54d-2f3ec8ac28d3" containerName="registry"
Jul 23 13:19:17.128727 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.128704    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.146345 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.146321    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-794f8595d4-ssdgk"]
Jul 23 13:19:17.208139 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-oauth-config\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.208139 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-oauth-serving-cert\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.208340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-config\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.208340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208250    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-serving-cert\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.208340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208307    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8mpk\" (UniqueName: \"kubernetes.io/projected/e160fa5d-6dce-4560-b32e-b1e14b2d024f-kube-api-access-k8mpk\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.208340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-service-ca\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.208509 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.208371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-trusted-ca-bundle\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309480 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-oauth-config\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309480 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-oauth-serving-cert\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309722 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-config\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309722 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-serving-cert\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309722 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309691    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8mpk\" (UniqueName: \"kubernetes.io/projected/e160fa5d-6dce-4560-b32e-b1e14b2d024f-kube-api-access-k8mpk\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309853 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-service-ca\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.309853 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.309758    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-trusted-ca-bundle\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.310402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.310349    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-config\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.310520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.310372    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-oauth-serving-cert\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.310596 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.310517    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-service-ca\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.310633 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.310611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-trusted-ca-bundle\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.312318 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.312292    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-oauth-config\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.312400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.312303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-serving-cert\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.319400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.319373    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8mpk\" (UniqueName: \"kubernetes.io/projected/e160fa5d-6dce-4560-b32e-b1e14b2d024f-kube-api-access-k8mpk\") pod \"console-794f8595d4-ssdgk\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") " pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.436967 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.436880    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:17.570851 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.570822    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-794f8595d4-ssdgk"]
Jul 23 13:19:17.573568 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:19:17.573527    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode160fa5d_6dce_4560_b32e_b1e14b2d024f.slice/crio-954589a34fc4a566602a9f504f921fd1c8b162578888f782ce1e20bc5e1fbc4b WatchSource:0}: Error finding container 954589a34fc4a566602a9f504f921fd1c8b162578888f782ce1e20bc5e1fbc4b: Status 404 returned error can't find the container with id 954589a34fc4a566602a9f504f921fd1c8b162578888f782ce1e20bc5e1fbc4b
Jul 23 13:19:17.827897 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.827872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:17.926102 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:17.926055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:18.574608 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:18.574574    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794f8595d4-ssdgk" event={"ID":"e160fa5d-6dce-4560-b32e-b1e14b2d024f","Type":"ContainerStarted","Data":"e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74"}
Jul 23 13:19:18.574768 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:18.574625    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794f8595d4-ssdgk" event={"ID":"e160fa5d-6dce-4560-b32e-b1e14b2d024f","Type":"ContainerStarted","Data":"954589a34fc4a566602a9f504f921fd1c8b162578888f782ce1e20bc5e1fbc4b"}
Jul 23 13:19:18.595009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:18.594966    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-794f8595d4-ssdgk" podStartSLOduration=1.5949522699999998 podStartE2EDuration="1.59495227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:19:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:19:18.593143874 +0000 UTC m=+88.969094650" watchObservedRunningTime="2026-07-23 13:19:18.59495227 +0000 UTC m=+88.970903051"
Jul 23 13:19:19.957456 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:19.957422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:20.060869 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:20.060821    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:20.584271 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:20.584242    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:20.584435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:20.584298    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:20.588887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:20.588867    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:20.962054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:20.961985    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:21.062027 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:21.062000    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:21.589352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:21.589328    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:22.116038 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:22.116011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:22.216848 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:22.216813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:22.317884 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:22.317850    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:25.849032 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:25.848989    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:25.953254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:25.953220    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:26.789086 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:26.789051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:26.853040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:26.853009    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:27.437941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.437906    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:27.438106 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.437967    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:27.442700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.442675    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:27.615122 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.615097    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:19:27.661447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.661406    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f9fbf744d-h5m56"]
Jul 23 13:19:27.765278 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.765191    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:27.827282 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.827250    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:27.906023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:27.905993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:31.296362 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:31.296329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:31.391773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:31.391740    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:35.503593 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:35.503560    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6bf7k"
Jul 23 13:19:35.948766 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:35.948729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:36.046913 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:36.046869    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:50.706429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:50.706394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:50.812921 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:50.812884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:52.682331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:52.682270    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f9fbf744d-h5m56" podUID="3505d1c8-f790-4e83-a417-a9e52f18a06d" containerName="console" containerID="cri-o://7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5" gracePeriod=15
Jul 23 13:19:52.926016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:52.925992    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:53.000048 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:52.999955    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-config\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000048 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000020    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-service-ca\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000054    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkmqm\" (UniqueName: \"kubernetes.io/projected/3505d1c8-f790-4e83-a417-a9e52f18a06d-kube-api-access-wkmqm\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000235    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-serving-cert\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000328 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000289    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-oauth-serving-cert\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000328 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000322    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-trusted-ca-bundle\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000370    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-oauth-config\") pod \"3505d1c8-f790-4e83-a417-a9e52f18a06d\" (UID: \"3505d1c8-f790-4e83-a417-a9e52f18a06d\") "
Jul 23 13:19:53.000475 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000414    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-config" (OuterVolumeSpecName: "console-config") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:19:53.000475 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000437    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-service-ca" (OuterVolumeSpecName: "service-ca") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:19:53.000669 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000625    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-service-ca\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.000669 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000646    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-config\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.000868 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000765    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:19:53.000868 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.000787    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:19:53.002416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.002391    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3505d1c8-f790-4e83-a417-a9e52f18a06d-kube-api-access-wkmqm" (OuterVolumeSpecName: "kube-api-access-wkmqm") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "kube-api-access-wkmqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:19:53.002501 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.002462    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:19:53.002622 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.002603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3505d1c8-f790-4e83-a417-a9e52f18a06d" (UID: "3505d1c8-f790-4e83-a417-a9e52f18a06d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:19:53.102023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.101991    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkmqm\" (UniqueName: \"kubernetes.io/projected/3505d1c8-f790-4e83-a417-a9e52f18a06d-kube-api-access-wkmqm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.102023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.102019    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-serving-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.102023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.102027    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-oauth-serving-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.102023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.102038    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3505d1c8-f790-4e83-a417-a9e52f18a06d-trusted-ca-bundle\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.102281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.102046    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3505d1c8-f790-4e83-a417-a9e52f18a06d-console-oauth-config\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:19:53.712154 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.712119    2559 generic.go:356] "Generic (PLEG): container finished" podID="3505d1c8-f790-4e83-a417-a9e52f18a06d" containerID="7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5" exitCode=0
Jul 23 13:19:53.712635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.712167    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9fbf744d-h5m56"
Jul 23 13:19:53.712635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.712193    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9fbf744d-h5m56" event={"ID":"3505d1c8-f790-4e83-a417-a9e52f18a06d","Type":"ContainerDied","Data":"7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5"}
Jul 23 13:19:53.712635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.712244    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9fbf744d-h5m56" event={"ID":"3505d1c8-f790-4e83-a417-a9e52f18a06d","Type":"ContainerDied","Data":"d3a9d6d4faa1490d2276d436afcd85994d62be746a956388d26b55d46439806c"}
Jul 23 13:19:53.712635 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.712281    2559 scope.go:117] "RemoveContainer" containerID="7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5"
Jul 23 13:19:53.721474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.721452    2559 scope.go:117] "RemoveContainer" containerID="7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5"
Jul 23 13:19:53.721774 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:19:53.721750    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5\": container with ID starting with 7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5 not found: ID does not exist" containerID="7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5"
Jul 23 13:19:53.721878 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.721782    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5"} err="failed to get container status \"7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5\": rpc error: code = NotFound desc = could not find container \"7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5\": container with ID starting with 7c10ce50f090a1401736b7481613beba6a9b4232d514074db10131587472b6f5 not found: ID does not exist"
Jul 23 13:19:53.737611 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.737583    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f9fbf744d-h5m56"]
Jul 23 13:19:53.741486 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:53.741463    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f9fbf744d-h5m56"]
Jul 23 13:19:54.090148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:54.090113    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3505d1c8-f790-4e83-a417-a9e52f18a06d" path="/var/lib/kubelet/pods/3505d1c8-f790-4e83-a417-a9e52f18a06d/volumes"
Jul 23 13:19:57.340215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:57.340174    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:57.399152 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:57.399115    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:19:57.456413 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:19:57.456385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:21.738682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:21.738644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:21.799363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:21.799330    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:30.089874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:30.089833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:31.203463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.203436    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f87f4d49b-4dcls"]
Jul 23 13:20:31.203844 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.203708    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3505d1c8-f790-4e83-a417-a9e52f18a06d" containerName="console"
Jul 23 13:20:31.203844 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.203720    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3505d1c8-f790-4e83-a417-a9e52f18a06d" containerName="console"
Jul 23 13:20:31.203844 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.203762    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3505d1c8-f790-4e83-a417-a9e52f18a06d" containerName="console"
Jul 23 13:20:31.205730 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.205714    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.223606 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.223582    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f87f4d49b-4dcls"]
Jul 23 13:20:31.280660 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.280622    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:31.294722 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-serving-cert\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.294824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-oauth-serving-cert\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.294824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294800    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-service-ca\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.294902 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-console-config\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.294902 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-trusted-ca-bundle\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.294902 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294871    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg887\" (UniqueName: \"kubernetes.io/projected/34e97e2b-7229-4311-9657-4e5c09043921-kube-api-access-fg887\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.295018 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.294917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-oauth-config\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.395988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-console-config\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.396021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-trusted-ca-bundle\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.396038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg887\" (UniqueName: \"kubernetes.io/projected/34e97e2b-7229-4311-9657-4e5c09043921-kube-api-access-fg887\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.396059    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-oauth-config\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.396122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-serving-cert\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.396148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-oauth-serving-cert\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.396234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.396175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-service-ca\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.397074 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.397045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-service-ca\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.397074 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.397043    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-oauth-serving-cert\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.397286 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.397048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-trusted-ca-bundle\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.397286 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.397080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-console-config\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.398791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.398765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-oauth-config\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.398896 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.398802    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-serving-cert\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.407664 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.407640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg887\" (UniqueName: \"kubernetes.io/projected/34e97e2b-7229-4311-9657-4e5c09043921-kube-api-access-fg887\") pod \"console-7f87f4d49b-4dcls\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") " pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.490186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.490091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:31.514557 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.514520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:31.650448 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.650422    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f87f4d49b-4dcls"]
Jul 23 13:20:31.653996 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:20:31.653954    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e97e2b_7229_4311_9657_4e5c09043921.slice/crio-31c0432bbfeadd84acbd9f6ccb7070d3a8f7949808a8c506ac0c3c5664193b73 WatchSource:0}: Error finding container 31c0432bbfeadd84acbd9f6ccb7070d3a8f7949808a8c506ac0c3c5664193b73: Status 404 returned error can't find the container with id 31c0432bbfeadd84acbd9f6ccb7070d3a8f7949808a8c506ac0c3c5664193b73
Jul 23 13:20:31.842876 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.842837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87f4d49b-4dcls" event={"ID":"34e97e2b-7229-4311-9657-4e5c09043921","Type":"ContainerStarted","Data":"ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401"}
Jul 23 13:20:31.843067 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:31.842914    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87f4d49b-4dcls" event={"ID":"34e97e2b-7229-4311-9657-4e5c09043921","Type":"ContainerStarted","Data":"31c0432bbfeadd84acbd9f6ccb7070d3a8f7949808a8c506ac0c3c5664193b73"}
Jul 23 13:20:33.410000 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:33.409967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:33.836812 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:33.836775    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:35.957845 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:35.957816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:36.058557 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:36.058507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:41.515464 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:41.515378    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:41.515464 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:41.515460    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:41.520047 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:41.520027    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:41.542914 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:41.542869    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f87f4d49b-4dcls" podStartSLOduration=10.542853907 podStartE2EDuration="10.542853907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:20:31.878983544 +0000 UTC m=+162.254934322" watchObservedRunningTime="2026-07-23 13:20:41.542853907 +0000 UTC m=+171.918804686"
Jul 23 13:20:41.881887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:41.881859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:20:41.935113 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:41.935082    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-794f8595d4-ssdgk"]
Jul 23 13:20:42.042174 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:42.042138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:42.105149 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:42.105110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:20:42.168196 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:20:42.168106    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:06.958355 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:06.958313    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-794f8595d4-ssdgk" podUID="e160fa5d-6dce-4560-b32e-b1e14b2d024f" containerName="console" containerID="cri-o://e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74" gracePeriod=15
Jul 23 13:21:07.191917 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.191896    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:21:07.294512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-trusted-ca-bundle\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.294512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294466    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-oauth-serving-cert\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.294512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294488    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-config\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.294512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294513    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-serving-cert\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.294843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294565    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8mpk\" (UniqueName: \"kubernetes.io/projected/e160fa5d-6dce-4560-b32e-b1e14b2d024f-kube-api-access-k8mpk\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.294843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-oauth-config\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.294843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294675    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-service-ca\") pod \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\" (UID: \"e160fa5d-6dce-4560-b32e-b1e14b2d024f\") "
Jul 23 13:21:07.295050 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294908    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-config" (OuterVolumeSpecName: "console-config") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:21:07.295050 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294919    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:21:07.295050 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.294941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:21:07.295180 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.295143    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-service-ca" (OuterVolumeSpecName: "service-ca") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:21:07.296929 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.296903    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:21:07.297036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.296965    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:21:07.297036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.296976    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e160fa5d-6dce-4560-b32e-b1e14b2d024f-kube-api-access-k8mpk" (OuterVolumeSpecName: "kube-api-access-k8mpk") pod "e160fa5d-6dce-4560-b32e-b1e14b2d024f" (UID: "e160fa5d-6dce-4560-b32e-b1e14b2d024f"). InnerVolumeSpecName "kube-api-access-k8mpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:21:07.395354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395310    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8mpk\" (UniqueName: \"kubernetes.io/projected/e160fa5d-6dce-4560-b32e-b1e14b2d024f-kube-api-access-k8mpk\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.395354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395347    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-oauth-config\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.395354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395357    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-service-ca\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.395354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395366    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-trusted-ca-bundle\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.395621 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395374    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-oauth-serving-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.395621 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395382    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-config\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.395621 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.395390    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e160fa5d-6dce-4560-b32e-b1e14b2d024f-console-serving-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:21:07.968153 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.968123    2559 generic.go:356] "Generic (PLEG): container finished" podID="e160fa5d-6dce-4560-b32e-b1e14b2d024f" containerID="e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74" exitCode=0
Jul 23 13:21:07.968588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.968170    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794f8595d4-ssdgk" event={"ID":"e160fa5d-6dce-4560-b32e-b1e14b2d024f","Type":"ContainerDied","Data":"e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74"}
Jul 23 13:21:07.968588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.968176    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-794f8595d4-ssdgk"
Jul 23 13:21:07.968588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.968213    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-794f8595d4-ssdgk" event={"ID":"e160fa5d-6dce-4560-b32e-b1e14b2d024f","Type":"ContainerDied","Data":"954589a34fc4a566602a9f504f921fd1c8b162578888f782ce1e20bc5e1fbc4b"}
Jul 23 13:21:07.968588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.968233    2559 scope.go:117] "RemoveContainer" containerID="e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74"
Jul 23 13:21:07.976667 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.976649    2559 scope.go:117] "RemoveContainer" containerID="e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74"
Jul 23 13:21:07.976924 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:21:07.976905    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74\": container with ID starting with e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74 not found: ID does not exist" containerID="e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74"
Jul 23 13:21:07.976972 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.976934    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74"} err="failed to get container status \"e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74\": rpc error: code = NotFound desc = could not find container \"e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74\": container with ID starting with e9f847fcf425f572923435a051d82e882b2c1fdd3a08196550e807f8ca8c9f74 not found: ID does not exist"
Jul 23 13:21:07.989527 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.989503    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-794f8595d4-ssdgk"]
Jul 23 13:21:07.992052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:07.992032    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-794f8595d4-ssdgk"]
Jul 23 13:21:08.090474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:08.090443    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e160fa5d-6dce-4560-b32e-b1e14b2d024f" path="/var/lib/kubelet/pods/e160fa5d-6dce-4560-b32e-b1e14b2d024f/volumes"
Jul 23 13:21:13.615825 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:13.615791    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:19.059596 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:19.059562    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:19.151599 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:19.151574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:23.484429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:23.484399    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:46.076072 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:46.076038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:47.461708 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:47.461667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:49.658879 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:49.658845    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:49.718747 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:49.718703    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:50.929417 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:50.929381    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:51.062443 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:51.062414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:51.204838 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:51.204753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:52.536315 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:52.536281    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:53.313992 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:53.313956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:56.460932 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:56.460900    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:21:56.560871 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:21:56.560838    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:02.117984 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:02.117951    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:04.884391 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:04.884347    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:04.985625 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:04.985590    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:06.402660 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:06.402630    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:06.503977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:06.503943    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:10.454001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:10.453969    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:10.555200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:10.555166    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:12.749320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:12.749243    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:28.294299 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:28.294270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:28.396747 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:28.396703    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:28.879739 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:28.879707    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.139852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.139758    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.242136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.242094    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.294590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.294559    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"]
Jul 23 13:22:29.294937 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.294799    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e160fa5d-6dce-4560-b32e-b1e14b2d024f" containerName="console"
Jul 23 13:22:29.294937 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.294810    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e160fa5d-6dce-4560-b32e-b1e14b2d024f" containerName="console"
Jul 23 13:22:29.294937 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.294862    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e160fa5d-6dce-4560-b32e-b1e14b2d024f" containerName="console"
Jul 23 13:22:29.297761 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.297742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.300850 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.300832    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 13:22:29.301055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.301036    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 13:22:29.301158 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.301036    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 13:22:29.301158 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.301099    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 13:22:29.301158 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.301041    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-msj2f\""
Jul 23 13:22:29.310973 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.310948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-manifests\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.311084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.311020    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-tmp\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.311084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.311066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.311200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.311095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.311200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.311122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4rb8\" (UniqueName: \"kubernetes.io/projected/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-kube-api-access-q4rb8\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.312293 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.312269    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"]
Jul 23 13:22:29.411749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.411665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-manifests\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.411749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.411700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-tmp\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.411749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.411723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.411749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.411740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.412079 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.411757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4rb8\" (UniqueName: \"kubernetes.io/projected/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-kube-api-access-q4rb8\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.412157 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.412134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-manifests\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.414307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.414277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-tmp\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.414435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.414418    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.414485 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.414466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.420608 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.420586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4rb8\" (UniqueName: \"kubernetes.io/projected/e0278557-d140-4f4b-92dc-f4dfe56bf6bd-kube-api-access-q4rb8\") pod \"opendatahub-operator-controller-manager-694f99b88-lrc5l\" (UID: \"e0278557-d140-4f4b-92dc-f4dfe56bf6bd\") " pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.432737 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.432699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.528714 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.528687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.607797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.607763    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:29.631518 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.631496    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.732924 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.732896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:29.743691 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.743667    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"]
Jul 23 13:22:29.746903 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:22:29.746882    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0278557_d140_4f4b_92dc_f4dfe56bf6bd.slice/crio-6a48680ced6a8ef2929d5f406cd0a7e28737cd8a97b67a67c433c809427d24f0 WatchSource:0}: Error finding container 6a48680ced6a8ef2929d5f406cd0a7e28737cd8a97b67a67c433c809427d24f0: Status 404 returned error can't find the container with id 6a48680ced6a8ef2929d5f406cd0a7e28737cd8a97b67a67c433c809427d24f0
Jul 23 13:22:29.837105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:29.837073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:30.235732 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:30.235697    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l" event={"ID":"e0278557-d140-4f4b-92dc-f4dfe56bf6bd","Type":"ContainerStarted","Data":"6a48680ced6a8ef2929d5f406cd0a7e28737cd8a97b67a67c433c809427d24f0"}
Jul 23 13:22:34.253136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:34.253095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l" event={"ID":"e0278557-d140-4f4b-92dc-f4dfe56bf6bd","Type":"ContainerStarted","Data":"13a773a01619684fb49f1ec95e56e31fba5bd1146dc90fee0f983023151f1ce9"}
Jul 23 13:22:35.257439 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:35.257398    2559 generic.go:356] "Generic (PLEG): container finished" podID="e0278557-d140-4f4b-92dc-f4dfe56bf6bd" containerID="13a773a01619684fb49f1ec95e56e31fba5bd1146dc90fee0f983023151f1ce9" exitCode=0
Jul 23 13:22:35.257852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:35.257448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l" event={"ID":"e0278557-d140-4f4b-92dc-f4dfe56bf6bd","Type":"ContainerDied","Data":"13a773a01619684fb49f1ec95e56e31fba5bd1146dc90fee0f983023151f1ce9"}
Jul 23 13:22:36.263041 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:36.262999    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l" event={"ID":"e0278557-d140-4f4b-92dc-f4dfe56bf6bd","Type":"ContainerStarted","Data":"55ae3643a6b3730323d0ddbc214e18ca48bee66af34a693e0f8192568cc89d09"}
Jul 23 13:22:36.263416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:36.263148    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:36.287244 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:36.287186    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l" podStartSLOduration=1.991004526 podStartE2EDuration="7.287167909s" totalImagesPullingTime="4.240032244s" totalInitContainerRuntime="1.056131139s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:22:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:22:36.285784066 +0000 UTC m=+286.661734845" watchObservedRunningTime="2026-07-23 13:22:36.287167909 +0000 UTC m=+286.663118685"
Jul 23 13:22:44.732394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:44.732353    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:47.269234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:47.269203    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-694f99b88-lrc5l"
Jul 23 13:22:47.612700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:47.612669    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:50.001804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:50.001778    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 13:22:50.951776 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:50.951747    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:51.492642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.492609    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:51.789272 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.789183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:51.855246 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.855211    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"]
Jul 23 13:22:51.857851 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.857832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:51.861792 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.861769    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-g44gr\""
Jul 23 13:22:51.863797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.863781    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 13:22:51.864614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.864462    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 13:22:51.864706 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.864688    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 13:22:51.864857 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.864838    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 13:22:51.977405 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.977376    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"]
Jul 23 13:22:51.995759 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.995719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5732263-83cd-4bf2-ba93-98c3d41589e6-tls-certs\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:51.995941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.995785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5732263-83cd-4bf2-ba93-98c3d41589e6-tmp\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:51.995941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:51.995870    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvhcb\" (UniqueName: \"kubernetes.io/projected/e5732263-83cd-4bf2-ba93-98c3d41589e6-kube-api-access-cvhcb\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.097015 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.096979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5732263-83cd-4bf2-ba93-98c3d41589e6-tls-certs\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.097214 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.097065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5732263-83cd-4bf2-ba93-98c3d41589e6-tmp\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.097214 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.097133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhcb\" (UniqueName: \"kubernetes.io/projected/e5732263-83cd-4bf2-ba93-98c3d41589e6-kube-api-access-cvhcb\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.097214 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:22:52.097159    2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 23 13:22:52.097390 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:22:52.097234    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5732263-83cd-4bf2-ba93-98c3d41589e6-tls-certs podName:e5732263-83cd-4bf2-ba93-98c3d41589e6 nodeName:}" failed. No retries permitted until 2026-07-23 13:22:52.597210225 +0000 UTC m=+302.973161004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e5732263-83cd-4bf2-ba93-98c3d41589e6-tls-certs") pod "kube-auth-proxy-75b4c49cfd-sf9xj" (UID: "e5732263-83cd-4bf2-ba93-98c3d41589e6") : secret "kube-auth-proxy-tls" not found
Jul 23 13:22:52.099421 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.099390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5732263-83cd-4bf2-ba93-98c3d41589e6-tmp\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.136423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.136381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvhcb\" (UniqueName: \"kubernetes.io/projected/e5732263-83cd-4bf2-ba93-98c3d41589e6-kube-api-access-cvhcb\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.228783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.228742    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hqh6w"]
Jul 23 13:22:52.232737 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.232707    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.236393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.236367    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 23 13:22:52.236510 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.236423    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pjv97\""
Jul 23 13:22:52.255896 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.255869    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hqh6w"]
Jul 23 13:22:52.299200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.299165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j82c6\" (UniqueName: \"kubernetes.io/projected/e0407af5-75a1-42ac-98ca-3d032303f02b-kube-api-access-j82c6\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.299363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.299272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0407af5-75a1-42ac-98ca-3d032303f02b-cert\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.400895 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.400789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0407af5-75a1-42ac-98ca-3d032303f02b-cert\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.400895 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.400890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j82c6\" (UniqueName: \"kubernetes.io/projected/e0407af5-75a1-42ac-98ca-3d032303f02b-kube-api-access-j82c6\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.401134 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:22:52.400978    2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 23 13:22:52.401134 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:22:52.401064    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0407af5-75a1-42ac-98ca-3d032303f02b-cert podName:e0407af5-75a1-42ac-98ca-3d032303f02b nodeName:}" failed. No retries permitted until 2026-07-23 13:22:52.901043551 +0000 UTC m=+303.276994321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e0407af5-75a1-42ac-98ca-3d032303f02b-cert") pod "odh-model-controller-7998c67dc4-hqh6w" (UID: "e0407af5-75a1-42ac-98ca-3d032303f02b") : secret "odh-model-controller-webhook-cert" not found
Jul 23 13:22:52.432818 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.432781    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j82c6\" (UniqueName: \"kubernetes.io/projected/e0407af5-75a1-42ac-98ca-3d032303f02b-kube-api-access-j82c6\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.480257 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.480214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:52.602390 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.602358    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5732263-83cd-4bf2-ba93-98c3d41589e6-tls-certs\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.605168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.605141    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5732263-83cd-4bf2-ba93-98c3d41589e6-tls-certs\") pod \"kube-auth-proxy-75b4c49cfd-sf9xj\" (UID: \"e5732263-83cd-4bf2-ba93-98c3d41589e6\") " pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.704982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.704903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:52.767105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.767074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"
Jul 23 13:22:52.901458 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.901425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:52.904508 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.904470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0407af5-75a1-42ac-98ca-3d032303f02b-cert\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.906971 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.906950    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0407af5-75a1-42ac-98ca-3d032303f02b-cert\") pod \"odh-model-controller-7998c67dc4-hqh6w\" (UID: \"e0407af5-75a1-42ac-98ca-3d032303f02b\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:52.911390 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.911364    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj"]
Jul 23 13:22:52.914615 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:22:52.914587    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5732263_83cd_4bf2_ba93_98c3d41589e6.slice/crio-e33e13344af123de1730e5572ab4dbb1ca624bb2644ad09efac012e6086e778a WatchSource:0}: Error finding container e33e13344af123de1730e5572ab4dbb1ca624bb2644ad09efac012e6086e778a: Status 404 returned error can't find the container with id e33e13344af123de1730e5572ab4dbb1ca624bb2644ad09efac012e6086e778a
Jul 23 13:22:52.916710 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:52.916691    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 13:22:53.143065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.143032    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:53.301302 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.301248    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hqh6w"]
Jul 23 13:22:53.303868 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:22:53.303830    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0407af5_75a1_42ac_98ca_3d032303f02b.slice/crio-b328256782ae11ab9a6607622ad57badac43ff06d8b428ad4145e2a87e7af4ad WatchSource:0}: Error finding container b328256782ae11ab9a6607622ad57badac43ff06d8b428ad4145e2a87e7af4ad: Status 404 returned error can't find the container with id b328256782ae11ab9a6607622ad57badac43ff06d8b428ad4145e2a87e7af4ad
Jul 23 13:22:53.324576 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.324527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w" event={"ID":"e0407af5-75a1-42ac-98ca-3d032303f02b","Type":"ContainerStarted","Data":"b328256782ae11ab9a6607622ad57badac43ff06d8b428ad4145e2a87e7af4ad"}
Jul 23 13:22:53.325714 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.325694    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj" event={"ID":"e5732263-83cd-4bf2-ba93-98c3d41589e6","Type":"ContainerStarted","Data":"e33e13344af123de1730e5572ab4dbb1ca624bb2644ad09efac012e6086e778a"}
Jul 23 13:22:53.923287 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.923257    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vb4bl"]
Jul 23 13:22:53.926591 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.926566    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:53.930657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.930629    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 13:22:53.930791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.930680    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-w9nwz\""
Jul 23 13:22:53.937820 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.937795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:53.949287 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:53.949228    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vb4bl"]
Jul 23 13:22:54.015372 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.015218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.015372 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.015281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x6rh\" (UniqueName: \"kubernetes.io/projected/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-kube-api-access-9x6rh\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.116396 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.116350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.116659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.116410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x6rh\" (UniqueName: \"kubernetes.io/projected/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-kube-api-access-9x6rh\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.116659 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:22:54.116638    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 23 13:22:54.116798 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:22:54.116704    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-tls-certs podName:d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d nodeName:}" failed. No retries permitted until 2026-07-23 13:22:54.616682599 +0000 UTC m=+304.992633361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-tls-certs") pod "model-serving-api-54d89bd5fc-vb4bl" (UID: "d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d") : secret "model-serving-api-tls" not found
Jul 23 13:22:54.146893 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.143479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x6rh\" (UniqueName: \"kubernetes.io/projected/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-kube-api-access-9x6rh\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.303005 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.302885    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:54.622079 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.622040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.630575 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.630488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vb4bl\" (UID: \"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:54.840707 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:54.840668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:55.090251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.090215    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"]
Jul 23 13:22:55.099933 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.099903    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.102926 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.102884    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pmglt\""
Jul 23 13:22:55.104380 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.104355    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 13:22:55.105025 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.104798    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 13:22:55.105025 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.104805    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 13:22:55.105423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.105321    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 13:22:55.113490 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.113463    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"]
Jul 23 13:22:55.172257 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.172226    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:55.229232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.229397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-cacerts\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.229397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2sfp\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-kube-api-access-l2sfp\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.229512 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.229593 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.229655 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ee8596-f1e3-4427-829e-4ce7518287d2-local-certs\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.229705 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.229648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-token\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.274606 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.274566    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:55.330632 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330595    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.330826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330659    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-cacerts\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.330826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2sfp\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-kube-api-access-l2sfp\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.330826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.330826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.330826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330816    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ee8596-f1e3-4427-829e-4ce7518287d2-local-certs\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.331090 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.330853    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-token\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.332086 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.331932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.334039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.333493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.334146 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.334109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.334952 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.334695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-cacerts\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.336015 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.335992    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ee8596-f1e3-4427-829e-4ce7518287d2-local-certs\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.339771 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.339748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-token\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.340787 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.340735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2sfp\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-kube-api-access-l2sfp\") pod \"istiod-openshift-gateway-7574459656-cqkz7\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:55.413827 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:55.413778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:22:56.750872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:56.750828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vb4bl"]
Jul 23 13:22:56.754669 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:22:56.754640    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e95b8e_86e1_4a6d_979c_556e8b5f8f1d.slice/crio-62eccaa02e67205d7ccd8a2a5d993491969aeaa63593a3f724b88923dbcfefa3 WatchSource:0}: Error finding container 62eccaa02e67205d7ccd8a2a5d993491969aeaa63593a3f724b88923dbcfefa3: Status 404 returned error can't find the container with id 62eccaa02e67205d7ccd8a2a5d993491969aeaa63593a3f724b88923dbcfefa3
Jul 23 13:22:56.771723 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:56.771704    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"]
Jul 23 13:22:56.774318 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:22:56.774291    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ee8596_f1e3_4427_829e_4ce7518287d2.slice/crio-6a67808ee9287e2f6b6f7b3cd4ac4b2a8df3d68eb46f0dc61bbaaea8327d850c WatchSource:0}: Error finding container 6a67808ee9287e2f6b6f7b3cd4ac4b2a8df3d68eb46f0dc61bbaaea8327d850c: Status 404 returned error can't find the container with id 6a67808ee9287e2f6b6f7b3cd4ac4b2a8df3d68eb46f0dc61bbaaea8327d850c
Jul 23 13:22:56.793973 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:56.793941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:22:57.344397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.344365    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" event={"ID":"36ee8596-f1e3-4427-829e-4ce7518287d2","Type":"ContainerStarted","Data":"6a67808ee9287e2f6b6f7b3cd4ac4b2a8df3d68eb46f0dc61bbaaea8327d850c"}
Jul 23 13:22:57.345766 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.345732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w" event={"ID":"e0407af5-75a1-42ac-98ca-3d032303f02b","Type":"ContainerStarted","Data":"b0d4172f07765ceca616e0c6fe31d18751beebc05a135a97f47457e73b915dd1"}
Jul 23 13:22:57.345894 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.345812    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:22:57.347269 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.347245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj" event={"ID":"e5732263-83cd-4bf2-ba93-98c3d41589e6","Type":"ContainerStarted","Data":"11ffa44b245fb76d59da5b9cc046cc8f388de4b759b9067fd2b54387809870ae"}
Jul 23 13:22:57.348313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.348282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl" event={"ID":"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d","Type":"ContainerStarted","Data":"62eccaa02e67205d7ccd8a2a5d993491969aeaa63593a3f724b88923dbcfefa3"}
Jul 23 13:22:57.367817 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.367779    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w" podStartSLOduration=2.038289944 podStartE2EDuration="5.367766638s" totalImagesPullingTime="3.329476694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:22:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:22:57.364945693 +0000 UTC m=+307.740896471" watchObservedRunningTime="2026-07-23 13:22:57.367766638 +0000 UTC m=+307.743717416"
Jul 23 13:22:57.386790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:57.386745    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75b4c49cfd-sf9xj" podStartSLOduration=2.720395257 podStartE2EDuration="6.386729738s" totalImagesPullingTime="3.666334481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:22:57.38457695 +0000 UTC m=+307.760527728" watchObservedRunningTime="2026-07-23 13:22:57.386729738 +0000 UTC m=+307.762680554"
Jul 23 13:22:59.360381 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:59.360349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl" event={"ID":"d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d","Type":"ContainerStarted","Data":"743a34462a3e3c59f129fae530b66b58a32f5386c335dd858ba3150fdc3dd54a"}
Jul 23 13:22:59.360877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:59.360465    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:22:59.379690 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:22:59.379639    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl" podStartSLOduration=4.516558231 podStartE2EDuration="6.379621042s" totalImagesPullingTime="1.863062811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:22:59.378466107 +0000 UTC m=+309.754416887" watchObservedRunningTime="2026-07-23 13:22:59.379621042 +0000 UTC m=+309.755571820"
Jul 23 13:23:01.022351 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:01.022314    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:23:01.022745 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:01.022399    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:23:01.371295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:01.371256    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" event={"ID":"36ee8596-f1e3-4427-829e-4ce7518287d2","Type":"ContainerStarted","Data":"168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b"}
Jul 23 13:23:01.371461 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:01.371335    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:23:01.497520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:01.497471    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" podStartSLOduration=2.251429148 podStartE2EDuration="6.497454344s" totalImagesPullingTime="4.246025196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:23:01.496851786 +0000 UTC m=+311.872802565" watchObservedRunningTime="2026-07-23 13:23:01.497454344 +0000 UTC m=+311.873405121"
Jul 23 13:23:01.827939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:01.827908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:02.008274 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:02.008233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:02.377065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:02.377029    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-cqkz7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 13:23:02.377423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:02.377079    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" podUID="36ee8596-f1e3-4427-829e-4ce7518287d2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 13:23:02.989682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:02.989645    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"]
Jul 23 13:23:02.993023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:02.992996    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:02.996638 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:02.996619    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-f5z6g\""
Jul 23 13:23:03.018332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.018301    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:03.042596 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.042566    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"]
Jul 23 13:23:03.107803 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.107750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.107803 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.107808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxkg9\" (UniqueName: \"kubernetes.io/projected/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-kube-api-access-qxkg9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.107867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.107897    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.107941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.107972    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.108011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.108135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.108323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.108171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209011 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.208973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209011 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxkg9\" (UniqueName: \"kubernetes.io/projected/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-kube-api-access-qxkg9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209236    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209616 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209716 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.209716 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.209660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.210106 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.210081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.211671 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.211652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.211844 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.211827    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.230185 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.230155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.230371 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.230347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxkg9\" (UniqueName: \"kubernetes.io/projected/7c6a1075-948d-4621-b0d8-ce4a65c52cdb-kube-api-access-qxkg9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fzhzk\" (UID: \"7c6a1075-948d-4621-b0d8-ce4a65c52cdb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.303270 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.303178    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:03.343367 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.343329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:03.494051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.494025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"]
Jul 23 13:23:03.496594 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:23:03.496568    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6a1075_948d_4621_b0d8_ce4a65c52cdb.slice/crio-d122ee3456ab7dd35d1fe4edaa3971506a739078a258ce317733bb0b4308b527 WatchSource:0}: Error finding container d122ee3456ab7dd35d1fe4edaa3971506a739078a258ce317733bb0b4308b527: Status 404 returned error can't find the container with id d122ee3456ab7dd35d1fe4edaa3971506a739078a258ce317733bb0b4308b527
Jul 23 13:23:03.723634 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:03.723602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:04.186648 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.186610    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:04.344913 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.344871    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"]
Jul 23 13:23:04.349220 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.349188    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.356693 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.356668    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"]
Jul 23 13:23:04.360168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.360149    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.360724 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.360706    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 23 13:23:04.360983 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.360969    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5chbq\""
Jul 23 13:23:04.378192 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.378166    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 13:23:04.378653 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.378637    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ffbwj\""
Jul 23 13:23:04.387183 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.387152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk" event={"ID":"7c6a1075-948d-4621-b0d8-ce4a65c52cdb","Type":"ContainerStarted","Data":"d122ee3456ab7dd35d1fe4edaa3971506a739078a258ce317733bb0b4308b527"}
Jul 23 13:23:04.400568 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.400523    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"]
Jul 23 13:23:04.401880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.401856    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"]
Jul 23 13:23:04.422773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.422742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:04.522650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.522563    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/077278b9-ac90-4eb0-9a15-915d41807e89-cert\") pod \"kserve-controller-manager-d6f4b65b8-vpdcd\" (UID: \"077278b9-ac90-4eb0-9a15-915d41807e89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.522650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.522617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17f827da-a6d2-4d6b-9fac-a13e854a6bb3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-prt2n\" (UID: \"17f827da-a6d2-4d6b-9fac-a13e854a6bb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.522650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.522643    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slmd4\" (UniqueName: \"kubernetes.io/projected/17f827da-a6d2-4d6b-9fac-a13e854a6bb3-kube-api-access-slmd4\") pod \"llmisvc-controller-manager-777c6bd7d5-prt2n\" (UID: \"17f827da-a6d2-4d6b-9fac-a13e854a6bb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.523073 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.522662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf8lz\" (UniqueName: \"kubernetes.io/projected/077278b9-ac90-4eb0-9a15-915d41807e89-kube-api-access-xf8lz\") pod \"kserve-controller-manager-d6f4b65b8-vpdcd\" (UID: \"077278b9-ac90-4eb0-9a15-915d41807e89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.621340 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.621281    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:04.623414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.623381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/077278b9-ac90-4eb0-9a15-915d41807e89-cert\") pod \"kserve-controller-manager-d6f4b65b8-vpdcd\" (UID: \"077278b9-ac90-4eb0-9a15-915d41807e89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.623570 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.623434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17f827da-a6d2-4d6b-9fac-a13e854a6bb3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-prt2n\" (UID: \"17f827da-a6d2-4d6b-9fac-a13e854a6bb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.623570 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.623462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slmd4\" (UniqueName: \"kubernetes.io/projected/17f827da-a6d2-4d6b-9fac-a13e854a6bb3-kube-api-access-slmd4\") pod \"llmisvc-controller-manager-777c6bd7d5-prt2n\" (UID: \"17f827da-a6d2-4d6b-9fac-a13e854a6bb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.623570 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.623492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf8lz\" (UniqueName: \"kubernetes.io/projected/077278b9-ac90-4eb0-9a15-915d41807e89-kube-api-access-xf8lz\") pod \"kserve-controller-manager-d6f4b65b8-vpdcd\" (UID: \"077278b9-ac90-4eb0-9a15-915d41807e89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.626583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.626558    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17f827da-a6d2-4d6b-9fac-a13e854a6bb3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-prt2n\" (UID: \"17f827da-a6d2-4d6b-9fac-a13e854a6bb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.626711 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.626596    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/077278b9-ac90-4eb0-9a15-915d41807e89-cert\") pod \"kserve-controller-manager-d6f4b65b8-vpdcd\" (UID: \"077278b9-ac90-4eb0-9a15-915d41807e89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.634370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.634155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf8lz\" (UniqueName: \"kubernetes.io/projected/077278b9-ac90-4eb0-9a15-915d41807e89-kube-api-access-xf8lz\") pod \"kserve-controller-manager-d6f4b65b8-vpdcd\" (UID: \"077278b9-ac90-4eb0-9a15-915d41807e89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.635187 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.635165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slmd4\" (UniqueName: \"kubernetes.io/projected/17f827da-a6d2-4d6b-9fac-a13e854a6bb3-kube-api-access-slmd4\") pod \"llmisvc-controller-manager-777c6bd7d5-prt2n\" (UID: \"17f827da-a6d2-4d6b-9fac-a13e854a6bb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.660887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.660849    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:04.669241 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.669213    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:04.751897 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.749991    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:04.863397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.863374    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"]
Jul 23 13:23:04.865299 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:23:04.865272    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077278b9_ac90_4eb0_9a15_915d41807e89.slice/crio-ce3d529adec8307dd5dbc00d18f0a019561c9a11d674d9b9d1c265307a565448 WatchSource:0}: Error finding container ce3d529adec8307dd5dbc00d18f0a019561c9a11d674d9b9d1c265307a565448: Status 404 returned error can't find the container with id ce3d529adec8307dd5dbc00d18f0a019561c9a11d674d9b9d1c265307a565448
Jul 23 13:23:04.882766 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:04.882726    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"]
Jul 23 13:23:04.885826 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:23:04.885795    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod17f827da_a6d2_4d6b_9fac_a13e854a6bb3.slice/crio-68ed2530a8ee8699731040fbf03ec5b7b560154154fea8f0bc8c07ad6de8be9f WatchSource:0}: Error finding container 68ed2530a8ee8699731040fbf03ec5b7b560154154fea8f0bc8c07ad6de8be9f: Status 404 returned error can't find the container with id 68ed2530a8ee8699731040fbf03ec5b7b560154154fea8f0bc8c07ad6de8be9f
Jul 23 13:23:05.376170 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:05.376137    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:23:05.393682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:05.393650    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd" event={"ID":"077278b9-ac90-4eb0-9a15-915d41807e89","Type":"ContainerStarted","Data":"ce3d529adec8307dd5dbc00d18f0a019561c9a11d674d9b9d1c265307a565448"}
Jul 23 13:23:05.395080 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:05.395057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n" event={"ID":"17f827da-a6d2-4d6b-9fac-a13e854a6bb3","Type":"ContainerStarted","Data":"68ed2530a8ee8699731040fbf03ec5b7b560154154fea8f0bc8c07ad6de8be9f"}
Jul 23 13:23:05.601914 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:05.601877    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:05.704460 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:05.704377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:05.796661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:05.796628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:06.100030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.099988    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"]
Jul 23 13:23:06.100268 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.100240    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" podUID="36ee8596-f1e3-4427-829e-4ce7518287d2" containerName="discovery" containerID="cri-o://168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b" gracePeriod=30
Jul 23 13:23:06.661673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.661636    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:23:06.662051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.661713    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:23:06.662051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.661742    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:23:06.768637 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.768343    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:23:06.947363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947276    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-kubeconfig\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.947363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947347    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ee8596-f1e3-4427-829e-4ce7518287d2-local-certs\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.947612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-cacerts\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.947612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947432    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-ca-configmap\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.947612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947497    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-dns-cert\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.947612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947527    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-token\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.947612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947584    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2sfp\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-kube-api-access-l2sfp\") pod \"36ee8596-f1e3-4427-829e-4ce7518287d2\" (UID: \"36ee8596-f1e3-4427-829e-4ce7518287d2\") "
Jul 23 13:23:06.948016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.947990    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:23:06.954465 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.951905    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-kube-api-access-l2sfp" (OuterVolumeSpecName: "kube-api-access-l2sfp") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "kube-api-access-l2sfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:23:06.954465 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.951987    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ee8596-f1e3-4427-829e-4ce7518287d2-local-certs" (OuterVolumeSpecName: "local-certs") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:23:06.954465 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.954456    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:23:06.954740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.954561    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:23:06.954740 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.954321    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:23:06.956400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:06.956332    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-cacerts" (OuterVolumeSpecName: "cacerts") pod "36ee8596-f1e3-4427-829e-4ce7518287d2" (UID: "36ee8596-f1e3-4427-829e-4ce7518287d2"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:23:07.048386 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048343    2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-cacerts\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.048386 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048381    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-ca-configmap\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.048659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048397    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-csr-dns-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.048659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048412    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.048659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048426    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2sfp\" (UniqueName: \"kubernetes.io/projected/36ee8596-f1e3-4427-829e-4ce7518287d2-kube-api-access-l2sfp\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.048659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048439    2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ee8596-f1e3-4427-829e-4ce7518287d2-istio-kubeconfig\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.048659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.048452    2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ee8596-f1e3-4427-829e-4ce7518287d2-local-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:07.407373 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.407325    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk" event={"ID":"7c6a1075-948d-4621-b0d8-ce4a65c52cdb","Type":"ContainerStarted","Data":"0d563f0e0c7309e632276a007a245c19d4da0afe35dfd95dcb7cca325efaf6d4"}
Jul 23 13:23:07.409781 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.409749    2559 generic.go:356] "Generic (PLEG): container finished" podID="36ee8596-f1e3-4427-829e-4ce7518287d2" containerID="168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b" exitCode=0
Jul 23 13:23:07.409909 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.409809    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" event={"ID":"36ee8596-f1e3-4427-829e-4ce7518287d2","Type":"ContainerDied","Data":"168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b"}
Jul 23 13:23:07.409909 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.409848    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7" event={"ID":"36ee8596-f1e3-4427-829e-4ce7518287d2","Type":"ContainerDied","Data":"6a67808ee9287e2f6b6f7b3cd4ac4b2a8df3d68eb46f0dc61bbaaea8327d850c"}
Jul 23 13:23:07.409909 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.409872    2559 scope.go:117] "RemoveContainer" containerID="168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b"
Jul 23 13:23:07.410058 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.410016    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"
Jul 23 13:23:07.444802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.444553    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk" podStartSLOduration=2.282198796 podStartE2EDuration="5.444509875s" totalImagesPullingTime="3.162311079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:23:07.441147656 +0000 UTC m=+317.817098435" watchObservedRunningTime="2026-07-23 13:23:07.444509875 +0000 UTC m=+317.820460654"
Jul 23 13:23:07.461852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.461811    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"]
Jul 23 13:23:07.468176 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.468138    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cqkz7"]
Jul 23 13:23:07.491081 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.491055    2559 scope.go:117] "RemoveContainer" containerID="168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b"
Jul 23 13:23:07.491570 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:23:07.491470    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b\": container with ID starting with 168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b not found: ID does not exist" containerID="168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b"
Jul 23 13:23:07.491570 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:07.491517    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b"} err="failed to get container status \"168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b\": rpc error: code = NotFound desc = could not find container \"168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b\": container with ID starting with 168e4f9247045482ce40262dda384588a52f1f839fc7f808697e16611a511e9b not found: ID does not exist"
Jul 23 13:23:08.092435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:08.092398    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36ee8596-f1e3-4427-829e-4ce7518287d2" path="/var/lib/kubelet/pods/36ee8596-f1e3-4427-829e-4ce7518287d2/volumes"
Jul 23 13:23:08.304038 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:08.303999    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:08.309907 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:08.309879    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:08.356650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:08.356572    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hqh6w"
Jul 23 13:23:08.418183 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:08.417748    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:08.418710 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:08.418649    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fzhzk"
Jul 23 13:23:09.423759 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:09.423727    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n" event={"ID":"17f827da-a6d2-4d6b-9fac-a13e854a6bb3","Type":"ContainerStarted","Data":"94e5984193828d52e2c1e9719b4039109c6082092de9c67d416b03aef91c1ad3"}
Jul 23 13:23:09.424244 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:09.423835    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:09.425367 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:09.425345    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd" event={"ID":"077278b9-ac90-4eb0-9a15-915d41807e89","Type":"ContainerStarted","Data":"9fdbbf50a78099decad1578c6e67aee50a37ed9dbee106ee253ac0d0f306724b"}
Jul 23 13:23:09.425548 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:09.425520    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:09.467873 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:09.467820    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n" podStartSLOduration=1.12491524 podStartE2EDuration="5.467804466s" totalImagesPullingTime="4.342889226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:23:09.464384491 +0000 UTC m=+319.840335294" watchObservedRunningTime="2026-07-23 13:23:09.467804466 +0000 UTC m=+319.843755243"
Jul 23 13:23:09.515256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:09.515142    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd" podStartSLOduration=1.151200915 podStartE2EDuration="5.515118689s" totalImagesPullingTime="4.363917774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:23:09.510390696 +0000 UTC m=+319.886341476" watchObservedRunningTime="2026-07-23 13:23:09.515118689 +0000 UTC m=+319.891069470"
Jul 23 13:23:10.369270 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:10.369242    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vb4bl"
Jul 23 13:23:39.077753 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:39.077721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:39.176270 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:39.176236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:40.432377 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:40.432348    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-prt2n"
Jul 23 13:23:40.435335 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:40.435316    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vpdcd"
Jul 23 13:23:49.377762 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:49.377662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:49.729457 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:49.729375    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:52.448965 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.448892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:52.616749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.616678    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cprdt"]
Jul 23 13:23:52.617132 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.617109    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36ee8596-f1e3-4427-829e-4ce7518287d2" containerName="discovery"
Jul 23 13:23:52.617132 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.617191    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="36ee8596-f1e3-4427-829e-4ce7518287d2" containerName="discovery"
Jul 23 13:23:52.617657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.617326    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="36ee8596-f1e3-4427-829e-4ce7518287d2" containerName="discovery"
Jul 23 13:23:52.620146 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.620128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:52.623002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.622978    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 13:23:52.624044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.624024    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-s5krw\""
Jul 23 13:23:52.624044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.624037    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 13:23:52.630875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.630850    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cprdt"]
Jul 23 13:23:52.650519 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.650495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lmb7\" (UniqueName: \"kubernetes.io/projected/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6-kube-api-access-9lmb7\") pod \"kuadrant-operator-catalog-cprdt\" (UID: \"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6\") " pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:52.751880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.751790    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lmb7\" (UniqueName: \"kubernetes.io/projected/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6-kube-api-access-9lmb7\") pod \"kuadrant-operator-catalog-cprdt\" (UID: \"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6\") " pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:52.760896 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.760864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lmb7\" (UniqueName: \"kubernetes.io/projected/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6-kube-api-access-9lmb7\") pod \"kuadrant-operator-catalog-cprdt\" (UID: \"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6\") " pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:52.930759 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.930723    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:52.974251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:52.974222    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cprdt"]
Jul 23 13:23:53.057741 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.057614    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cprdt"]
Jul 23 13:23:53.059975 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:23:53.059943    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bae22a4_d3b3_4f75_a972_e53d1baf0ac6.slice/crio-6b29f38ed1815a6599095791a017898455eb72b38106ba218d21d66fa2315e90 WatchSource:0}: Error finding container 6b29f38ed1815a6599095791a017898455eb72b38106ba218d21d66fa2315e90: Status 404 returned error can't find the container with id 6b29f38ed1815a6599095791a017898455eb72b38106ba218d21d66fa2315e90
Jul 23 13:23:53.183567 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.183515    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4wh79"]
Jul 23 13:23:53.186549 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.186511    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:23:53.194466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.194441    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4wh79"]
Jul 23 13:23:53.256717 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.256677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9h5r\" (UniqueName: \"kubernetes.io/projected/9deb41aa-aafe-4dc1-86ef-45eb850eb3f4-kube-api-access-m9h5r\") pod \"kuadrant-operator-catalog-4wh79\" (UID: \"9deb41aa-aafe-4dc1-86ef-45eb850eb3f4\") " pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:23:53.357919 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.357878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9h5r\" (UniqueName: \"kubernetes.io/projected/9deb41aa-aafe-4dc1-86ef-45eb850eb3f4-kube-api-access-m9h5r\") pod \"kuadrant-operator-catalog-4wh79\" (UID: \"9deb41aa-aafe-4dc1-86ef-45eb850eb3f4\") " pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:23:53.367580 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.367552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9h5r\" (UniqueName: \"kubernetes.io/projected/9deb41aa-aafe-4dc1-86ef-45eb850eb3f4-kube-api-access-m9h5r\") pod \"kuadrant-operator-catalog-4wh79\" (UID: \"9deb41aa-aafe-4dc1-86ef-45eb850eb3f4\") " pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:23:53.496212 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.496179    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:23:53.609830 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.609755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cprdt" event={"ID":"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6","Type":"ContainerStarted","Data":"6b29f38ed1815a6599095791a017898455eb72b38106ba218d21d66fa2315e90"}
Jul 23 13:23:53.623485 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:53.623455    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4wh79"]
Jul 23 13:23:53.626279 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:23:53.626251    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9deb41aa_aafe_4dc1_86ef_45eb850eb3f4.slice/crio-1f3abb1b9d52b7a5559890b09b499a75a88fb8f3c65c8e6f7158b399b4651e3a WatchSource:0}: Error finding container 1f3abb1b9d52b7a5559890b09b499a75a88fb8f3c65c8e6f7158b399b4651e3a: Status 404 returned error can't find the container with id 1f3abb1b9d52b7a5559890b09b499a75a88fb8f3c65c8e6f7158b399b4651e3a
Jul 23 13:23:54.509517 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:54.509473    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:54.610945 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:54.610872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:23:54.617932 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:54.617858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4wh79" event={"ID":"9deb41aa-aafe-4dc1-86ef-45eb850eb3f4","Type":"ContainerStarted","Data":"1f3abb1b9d52b7a5559890b09b499a75a88fb8f3c65c8e6f7158b399b4651e3a"}
Jul 23 13:23:55.624592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:55.624564    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4wh79" event={"ID":"9deb41aa-aafe-4dc1-86ef-45eb850eb3f4","Type":"ContainerStarted","Data":"b1efe88e5bc57c9adb679ab1764ac9b23392220a45a61007ad97d15a692d1003"}
Jul 23 13:23:55.647561 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:55.643022    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4wh79" podStartSLOduration=0.767365646 podStartE2EDuration="2.642992554s" totalImagesPullingTime="1.875626908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:23:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:23:55.640621878 +0000 UTC m=+366.016572657" watchObservedRunningTime="2026-07-23 13:23:55.642992554 +0000 UTC m=+366.018943333"
Jul 23 13:23:56.629702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.629663    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cprdt" event={"ID":"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6","Type":"ContainerStarted","Data":"2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef"}
Jul 23 13:23:56.630150 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.629718    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-cprdt" podUID="3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" containerName="registry-server" containerID="cri-o://2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef" gracePeriod=2
Jul 23 13:23:56.645867 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.645824    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cprdt" podStartSLOduration=2.106206612 podStartE2EDuration="4.645808582s" totalImagesPullingTime="2.53960197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:23:56.645035184 +0000 UTC m=+367.020985961" watchObservedRunningTime="2026-07-23 13:23:56.645808582 +0000 UTC m=+367.021759341"
Jul 23 13:23:56.870965 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.870943    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:56.891940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.891876    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lmb7\" (UniqueName: \"kubernetes.io/projected/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6-kube-api-access-9lmb7\") pod \"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6\" (UID: \"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6\") "
Jul 23 13:23:56.894317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.894290    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6-kube-api-access-9lmb7" (OuterVolumeSpecName: "kube-api-access-9lmb7") pod "3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" (UID: "3bae22a4-d3b3-4f75-a972-e53d1baf0ac6"). InnerVolumeSpecName "kube-api-access-9lmb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:23:56.993188 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:56.993151    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lmb7\" (UniqueName: \"kubernetes.io/projected/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6-kube-api-access-9lmb7\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:23:57.637892 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.637854    2559 generic.go:356] "Generic (PLEG): container finished" podID="3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" containerID="2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef" exitCode=0
Jul 23 13:23:57.638361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.637897    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cprdt"
Jul 23 13:23:57.638361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.637938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cprdt" event={"ID":"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6","Type":"ContainerDied","Data":"2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef"}
Jul 23 13:23:57.638361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.637998    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cprdt" event={"ID":"3bae22a4-d3b3-4f75-a972-e53d1baf0ac6","Type":"ContainerDied","Data":"6b29f38ed1815a6599095791a017898455eb72b38106ba218d21d66fa2315e90"}
Jul 23 13:23:57.638361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.638026    2559 scope.go:117] "RemoveContainer" containerID="2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef"
Jul 23 13:23:57.646951 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.646933    2559 scope.go:117] "RemoveContainer" containerID="2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef"
Jul 23 13:23:57.647206 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:23:57.647190    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef\": container with ID starting with 2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef not found: ID does not exist" containerID="2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef"
Jul 23 13:23:57.647253 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.647214    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef"} err="failed to get container status \"2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef\": rpc error: code = NotFound desc = could not find container \"2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef\": container with ID starting with 2c2c38b90ba72e5cb7265a790a26401172abaf2ee7d21cb66c2a61930df98eef not found: ID does not exist"
Jul 23 13:23:57.659483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.659463    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cprdt"]
Jul 23 13:23:57.663117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:57.663099    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cprdt"]
Jul 23 13:23:58.091238 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:23:58.091207    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" path="/var/lib/kubelet/pods/3bae22a4-d3b3-4f75-a972-e53d1baf0ac6/volumes"
Jul 23 13:24:03.496607 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:03.496571    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:24:03.497100 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:03.496643    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:24:03.518847 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:03.518823    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:24:03.686684 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:03.686660    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4wh79"
Jul 23 13:24:16.296774 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:16.296737    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:16.933741 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:16.933700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:17.943912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:17.943877    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:18.820763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.820729    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9c8fff85b-nsnh7"]
Jul 23 13:24:18.821111 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.821092    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" containerName="registry-server"
Jul 23 13:24:18.821200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.821113    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" containerName="registry-server"
Jul 23 13:24:18.821265 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.821201    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bae22a4-d3b3-4f75-a972-e53d1baf0ac6" containerName="registry-server"
Jul 23 13:24:18.824191 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.824169    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.836110 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.836086    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c8fff85b-nsnh7"]
Jul 23 13:24:18.928645 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.928612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:18.977080 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-console-config\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.977483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977107    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-oauth-serving-cert\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.977483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3289aa17-967a-43b2-aec8-6d9de1f45656-console-oauth-config\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.977483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3289aa17-967a-43b2-aec8-6d9de1f45656-console-serving-cert\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.977483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-trusted-ca-bundle\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.977483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-service-ca\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:18.977483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:18.977370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5zg\" (UniqueName: \"kubernetes.io/projected/3289aa17-967a-43b2-aec8-6d9de1f45656-kube-api-access-2h5zg\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.021686 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.021658    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:19.077808 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.077781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-console-config\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.077934 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.077828    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-oauth-serving-cert\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.077992 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.077952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3289aa17-967a-43b2-aec8-6d9de1f45656-console-oauth-config\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3289aa17-967a-43b2-aec8-6d9de1f45656-console-serving-cert\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078087 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-trusted-ca-bundle\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-service-ca\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078109    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5zg\" (UniqueName: \"kubernetes.io/projected/3289aa17-967a-43b2-aec8-6d9de1f45656-kube-api-access-2h5zg\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078689 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-console-config\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-oauth-serving-cert\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078884    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-service-ca\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.078984 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.078969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3289aa17-967a-43b2-aec8-6d9de1f45656-trusted-ca-bundle\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.080609 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.080588    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3289aa17-967a-43b2-aec8-6d9de1f45656-console-serving-cert\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.080731 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.080711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3289aa17-967a-43b2-aec8-6d9de1f45656-console-oauth-config\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.097934 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.097913    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5zg\" (UniqueName: \"kubernetes.io/projected/3289aa17-967a-43b2-aec8-6d9de1f45656-kube-api-access-2h5zg\") pod \"console-9c8fff85b-nsnh7\" (UID: \"3289aa17-967a-43b2-aec8-6d9de1f45656\") " pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.121758 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.121732    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:19.133808 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.133786    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:19.264782 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.264759    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c8fff85b-nsnh7"]
Jul 23 13:24:19.266987 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:24:19.266960    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3289aa17_967a_43b2_aec8_6d9de1f45656.slice/crio-7f0026d7eaedb45816075d3c074657d3f95e5c526ad1eb5a426505d7a7e3c347 WatchSource:0}: Error finding container 7f0026d7eaedb45816075d3c074657d3f95e5c526ad1eb5a426505d7a7e3c347: Status 404 returned error can't find the container with id 7f0026d7eaedb45816075d3c074657d3f95e5c526ad1eb5a426505d7a7e3c347
Jul 23 13:24:19.619206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.619114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:19.720405 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.720375    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:19.729131 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.729096    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c8fff85b-nsnh7" event={"ID":"3289aa17-967a-43b2-aec8-6d9de1f45656","Type":"ContainerStarted","Data":"9c3b3b574f43b360e6728ae0cefde0803e5849281db7ce06db7ad85aa9695435"}
Jul 23 13:24:19.729274 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.729138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c8fff85b-nsnh7" event={"ID":"3289aa17-967a-43b2-aec8-6d9de1f45656","Type":"ContainerStarted","Data":"7f0026d7eaedb45816075d3c074657d3f95e5c526ad1eb5a426505d7a7e3c347"}
Jul 23 13:24:19.748860 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.748817    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9c8fff85b-nsnh7" podStartSLOduration=1.748803611 podStartE2EDuration="1.748803611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:24:19.74748614 +0000 UTC m=+390.123436930" watchObservedRunningTime="2026-07-23 13:24:19.748803611 +0000 UTC m=+390.124754389"
Jul 23 13:24:19.965287 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.965214    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"]
Jul 23 13:24:19.968591 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.968575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:19.971504 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.971485    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sx845\""
Jul 23 13:24:19.983744 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:19.983722    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"]
Jul 23 13:24:20.087021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:20.086987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqnmv\" (UniqueName: \"kubernetes.io/projected/983c0ead-ef43-4c09-a525-b789cf4ca0f5-kube-api-access-vqnmv\") pod \"authorino-operator-75cc8d5cb7-2hvkb\" (UID: \"983c0ead-ef43-4c09-a525-b789cf4ca0f5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:20.187585 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:20.187557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqnmv\" (UniqueName: \"kubernetes.io/projected/983c0ead-ef43-4c09-a525-b789cf4ca0f5-kube-api-access-vqnmv\") pod \"authorino-operator-75cc8d5cb7-2hvkb\" (UID: \"983c0ead-ef43-4c09-a525-b789cf4ca0f5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:20.198914 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:20.198882    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqnmv\" (UniqueName: \"kubernetes.io/projected/983c0ead-ef43-4c09-a525-b789cf4ca0f5-kube-api-access-vqnmv\") pod \"authorino-operator-75cc8d5cb7-2hvkb\" (UID: \"983c0ead-ef43-4c09-a525-b789cf4ca0f5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:20.277986 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:20.277922    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:20.414828 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:20.414805    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"]
Jul 23 13:24:20.416808 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:24:20.416781    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983c0ead_ef43_4c09_a525_b789cf4ca0f5.slice/crio-4087f7fd3d7ce15eebaffef881fd83e7c1c2fefb67c07ae5299bbcb57fcb850e WatchSource:0}: Error finding container 4087f7fd3d7ce15eebaffef881fd83e7c1c2fefb67c07ae5299bbcb57fcb850e: Status 404 returned error can't find the container with id 4087f7fd3d7ce15eebaffef881fd83e7c1c2fefb67c07ae5299bbcb57fcb850e
Jul 23 13:24:20.734455 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:20.734424    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb" event={"ID":"983c0ead-ef43-4c09-a525-b789cf4ca0f5","Type":"ContainerStarted","Data":"4087f7fd3d7ce15eebaffef881fd83e7c1c2fefb67c07ae5299bbcb57fcb850e"}
Jul 23 13:24:22.513520 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:22.513489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:25.018394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.018354    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"]
Jul 23 13:24:25.028508 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.028484    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:25.032404 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.032379    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fm8r7\""
Jul 23 13:24:25.033897 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.033869    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"]
Jul 23 13:24:25.132388 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.132351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nqpt\" (UniqueName: \"kubernetes.io/projected/8539726f-a4c9-41ed-bf95-202f1c1ec016-kube-api-access-2nqpt\") pod \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" (UID: \"8539726f-a4c9-41ed-bf95-202f1c1ec016\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:25.233223 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.233149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nqpt\" (UniqueName: \"kubernetes.io/projected/8539726f-a4c9-41ed-bf95-202f1c1ec016-kube-api-access-2nqpt\") pod \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" (UID: \"8539726f-a4c9-41ed-bf95-202f1c1ec016\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:25.245505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.245470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nqpt\" (UniqueName: \"kubernetes.io/projected/8539726f-a4c9-41ed-bf95-202f1c1ec016-kube-api-access-2nqpt\") pod \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" (UID: \"8539726f-a4c9-41ed-bf95-202f1c1ec016\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:25.340065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.340032    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:25.503944 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.503872    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"]
Jul 23 13:24:25.506722 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:24:25.506698    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8539726f_a4c9_41ed_bf95_202f1c1ec016.slice/crio-2e29407ab45b769925050ceb03a912d49bda4218395a873d14d85c7b47395b95 WatchSource:0}: Error finding container 2e29407ab45b769925050ceb03a912d49bda4218395a873d14d85c7b47395b95: Status 404 returned error can't find the container with id 2e29407ab45b769925050ceb03a912d49bda4218395a873d14d85c7b47395b95
Jul 23 13:24:25.758285 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.758187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb" event={"ID":"983c0ead-ef43-4c09-a525-b789cf4ca0f5","Type":"ContainerStarted","Data":"b3b9347d38a12bf07129d6258ef05e6bd0968031efce0b2fd5bd5761373818db"}
Jul 23 13:24:25.758285 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.758250    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:25.759405 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.759386    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd" event={"ID":"8539726f-a4c9-41ed-bf95-202f1c1ec016","Type":"ContainerStarted","Data":"2e29407ab45b769925050ceb03a912d49bda4218395a873d14d85c7b47395b95"}
Jul 23 13:24:25.782999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:25.782956    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb" podStartSLOduration=1.828409654 podStartE2EDuration="6.782943404s" totalImagesPullingTime="4.95453375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:24:25.78194142 +0000 UTC m=+396.157892199" watchObservedRunningTime="2026-07-23 13:24:25.782943404 +0000 UTC m=+396.158894247"
Jul 23 13:24:26.123162 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:26.123135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:27.771084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:27.771045    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd" event={"ID":"8539726f-a4c9-41ed-bf95-202f1c1ec016","Type":"ContainerStarted","Data":"b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69"}
Jul 23 13:24:27.771436 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:27.771115    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:27.815823 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:27.815774    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd" podStartSLOduration=2.353624474 podStartE2EDuration="3.815758526s" totalImagesPullingTime="1.462134052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:24:27.813916449 +0000 UTC m=+398.189867227" watchObservedRunningTime="2026-07-23 13:24:27.815758526 +0000 UTC m=+398.191709304"
Jul 23 13:24:28.173808 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.173776    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"]
Jul 23 13:24:28.177180 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.177152    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:28.180138 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.180119    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 13:24:28.180855 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.180839    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hhdj7\""
Jul 23 13:24:28.191650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.191626    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"]
Jul 23 13:24:28.264084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.264055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wwb\" (UniqueName: \"kubernetes.io/projected/f4c71fd8-40fe-4a40-82be-3ce99a9b4050-kube-api-access-q7wwb\") pod \"dns-operator-controller-manager-864dd78c4c-f54gc\" (UID: \"f4c71fd8-40fe-4a40-82be-3ce99a9b4050\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:28.365391 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.365359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wwb\" (UniqueName: \"kubernetes.io/projected/f4c71fd8-40fe-4a40-82be-3ce99a9b4050-kube-api-access-q7wwb\") pod \"dns-operator-controller-manager-864dd78c4c-f54gc\" (UID: \"f4c71fd8-40fe-4a40-82be-3ce99a9b4050\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:28.383233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.383206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wwb\" (UniqueName: \"kubernetes.io/projected/f4c71fd8-40fe-4a40-82be-3ce99a9b4050-kube-api-access-q7wwb\") pod \"dns-operator-controller-manager-864dd78c4c-f54gc\" (UID: \"f4c71fd8-40fe-4a40-82be-3ce99a9b4050\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:28.487237 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.487147    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:28.620690 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.620667    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"]
Jul 23 13:24:28.623295 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:24:28.623254    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c71fd8_40fe_4a40_82be_3ce99a9b4050.slice/crio-6643fff2a4453b6efcd3777d9818224d751c6343b2e0f46813a3b73bc001cc0e WatchSource:0}: Error finding container 6643fff2a4453b6efcd3777d9818224d751c6343b2e0f46813a3b73bc001cc0e: Status 404 returned error can't find the container with id 6643fff2a4453b6efcd3777d9818224d751c6343b2e0f46813a3b73bc001cc0e
Jul 23 13:24:28.710354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.710321    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:28.776777 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:28.776691    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc" event={"ID":"f4c71fd8-40fe-4a40-82be-3ce99a9b4050","Type":"ContainerStarted","Data":"6643fff2a4453b6efcd3777d9818224d751c6343b2e0f46813a3b73bc001cc0e"}
Jul 23 13:24:29.134794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.134760    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:29.134990 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.134849    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:29.139639 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.139621    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:29.747801 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.747760    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:29.786423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.786395    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9c8fff85b-nsnh7"
Jul 23 13:24:29.850044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.850004    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f87f4d49b-4dcls"]
Jul 23 13:24:29.984166 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:29.984128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:30.091119 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:30.091085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:30.185993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:30.185946    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:30.787462 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:30.787381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc" event={"ID":"f4c71fd8-40fe-4a40-82be-3ce99a9b4050","Type":"ContainerStarted","Data":"954d09b7ceba34a93ea7827be41f97c7c49609ac5a6f5ad1ab7e89de36974b8a"}
Jul 23 13:24:30.787832 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:30.787486    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:30.812598 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:30.812528    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc" podStartSLOduration=0.930711491 podStartE2EDuration="2.812508034s" totalImagesPullingTime="1.881796543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:24:30.810858899 +0000 UTC m=+401.186809677" watchObservedRunningTime="2026-07-23 13:24:30.812508034 +0000 UTC m=+401.188458813"
Jul 23 13:24:30.973192 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:30.973159    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:32.751055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:32.751026    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:34.544089 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:34.544050    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:34.650186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:34.650157    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:34.919878 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:34.919843    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:35.016156 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:35.016121    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:36.768259 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:36.768225    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2hvkb"
Jul 23 13:24:38.559690 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:38.559658    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:38.715474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:38.715441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:38.779482 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:38.779454    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:40.713241 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.713209    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"]
Jul 23 13:24:40.716418 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.716399    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.716989 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.716963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:40.718925 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.718899    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 13:24:40.719043 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.718926    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6cp99\""
Jul 23 13:24:40.719043 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.718900    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 13:24:40.723002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.722981    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"]
Jul 23 13:24:40.766559 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.766497    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/152956f6-7878-40e9-ac9e-405dc84d1eec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.766718 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.766576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvsj9\" (UniqueName: \"kubernetes.io/projected/152956f6-7878-40e9-ac9e-405dc84d1eec-kube-api-access-wvsj9\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.766718 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.766671    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/152956f6-7878-40e9-ac9e-405dc84d1eec-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.867376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.867345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvsj9\" (UniqueName: \"kubernetes.io/projected/152956f6-7878-40e9-ac9e-405dc84d1eec-kube-api-access-wvsj9\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.867565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.867414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/152956f6-7878-40e9-ac9e-405dc84d1eec-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.867565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.867492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/152956f6-7878-40e9-ac9e-405dc84d1eec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.867660 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:24:40.867624    2559 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 23 13:24:40.867706 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:24:40.867696    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152956f6-7878-40e9-ac9e-405dc84d1eec-plugin-serving-cert podName:152956f6-7878-40e9-ac9e-405dc84d1eec nodeName:}" failed. No retries permitted until 2026-07-23 13:24:41.367674965 +0000 UTC m=+411.743625727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/152956f6-7878-40e9-ac9e-405dc84d1eec-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-wr9kv" (UID: "152956f6-7878-40e9-ac9e-405dc84d1eec") : secret "plugin-serving-cert" not found
Jul 23 13:24:40.868072 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.868053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/152956f6-7878-40e9-ac9e-405dc84d1eec-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.878819 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.878794    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvsj9\" (UniqueName: \"kubernetes.io/projected/152956f6-7878-40e9-ac9e-405dc84d1eec-kube-api-access-wvsj9\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:40.982815 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:40.982736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:41.370680 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.370646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/152956f6-7878-40e9-ac9e-405dc84d1eec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:41.373166 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.373146    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/152956f6-7878-40e9-ac9e-405dc84d1eec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wr9kv\" (UID: \"152956f6-7878-40e9-ac9e-405dc84d1eec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:41.631272 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.631170    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"
Jul 23 13:24:41.768977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.768949    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv"]
Jul 23 13:24:41.771184 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:24:41.771160    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod152956f6_7878_40e9_ac9e_405dc84d1eec.slice/crio-80a096905ec45f3e77aad7318c81f9287b791299f3284e47b9a74f6e1251f64e WatchSource:0}: Error finding container 80a096905ec45f3e77aad7318c81f9287b791299f3284e47b9a74f6e1251f64e: Status 404 returned error can't find the container with id 80a096905ec45f3e77aad7318c81f9287b791299f3284e47b9a74f6e1251f64e
Jul 23 13:24:41.796481 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.796461    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f54gc"
Jul 23 13:24:41.842026 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.841988    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv" event={"ID":"152956f6-7878-40e9-ac9e-405dc84d1eec","Type":"ContainerStarted","Data":"80a096905ec45f3e77aad7318c81f9287b791299f3284e47b9a74f6e1251f64e"}
Jul 23 13:24:41.947979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:41.947895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:42.034134 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:42.034100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:45.669762 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:45.669721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:45.770022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:45.769981    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:51.353169 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.353132    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"]
Jul 23 13:24:51.353696 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.353399    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" containerName="manager" containerID="cri-o://b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69" gracePeriod=2
Jul 23 13:24:51.362717 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.361011    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"]
Jul 23 13:24:51.379874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.379847    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"]
Jul 23 13:24:51.380319 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.380289    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" containerName="manager"
Jul 23 13:24:51.380319 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.380313    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" containerName="manager"
Jul 23 13:24:51.380441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.380401    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" containerName="manager"
Jul 23 13:24:51.392250 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.392224    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:24:51.398382 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:24:51.398342    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" is forbidden: User \"system:node:ip-10-0-136-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-232.ec2.internal' and this object" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:24:51.400296 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.400269    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"]
Jul 23 13:24:51.471786 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.471732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz9pt\" (UniqueName: \"kubernetes.io/projected/e95dccf5-4c7d-4304-871d-ced1afe59c2f-kube-api-access-sz9pt\") pod \"limitador-operator-controller-manager-859d7b78bd-rxjs7\" (UID: \"e95dccf5-4c7d-4304-871d-ced1afe59c2f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:24:51.572559 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.572499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz9pt\" (UniqueName: \"kubernetes.io/projected/e95dccf5-4c7d-4304-871d-ced1afe59c2f-kube-api-access-sz9pt\") pod \"limitador-operator-controller-manager-859d7b78bd-rxjs7\" (UID: \"e95dccf5-4c7d-4304-871d-ced1afe59c2f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:24:51.586599 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.586569    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz9pt\" (UniqueName: \"kubernetes.io/projected/e95dccf5-4c7d-4304-871d-ced1afe59c2f-kube-api-access-sz9pt\") pod \"limitador-operator-controller-manager-859d7b78bd-rxjs7\" (UID: \"e95dccf5-4c7d-4304-871d-ced1afe59c2f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:24:51.737262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:51.737182    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:24:52.520979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:52.520949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:52.615924 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:52.615892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:53.526360 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:53.526324    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:53.625937 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:53.625907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:53.920128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:53.920096    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:54.019200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:54.019163    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:54.872103 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:54.872065    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f87f4d49b-4dcls" podUID="34e97e2b-7229-4311-9657-4e5c09043921" containerName="console" containerID="cri-o://ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401" gracePeriod=15
Jul 23 13:24:57.508442 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:57.508404    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:24:57.609826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:24:57.609170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:00.111195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:00.111155    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:00.172627 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:00.172591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:01.878907 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:01.878861    2559 patch_prober.go:28] interesting pod/console-7f87f4d49b-4dcls container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" start-of-body=
Jul 23 13:25:01.879280 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:01.878934    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-7f87f4d49b-4dcls" podUID="34e97e2b-7229-4311-9657-4e5c09043921" containerName="console" probeResult="failure" output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused"
Jul 23 13:25:02.772312 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.772235    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"]
Jul 23 13:25:02.786841 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:25:02.786813    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95dccf5_4c7d_4304_871d_ced1afe59c2f.slice/crio-e82e48318e7e3bc0134a99952e1bb207bba1526ad4be1e21aa1afb768e0c49d3 WatchSource:0}: Error finding container e82e48318e7e3bc0134a99952e1bb207bba1526ad4be1e21aa1afb768e0c49d3: Status 404 returned error can't find the container with id e82e48318e7e3bc0134a99952e1bb207bba1526ad4be1e21aa1afb768e0c49d3
Jul 23 13:25:02.794614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.793080    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:25:02.823424 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.823400    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:25:02.825855 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:02.825816    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" is forbidden: User \"system:node:ip-10-0-136-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-232.ec2.internal' and this object" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:25:02.873521 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873461    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-trusted-ca-bundle\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.873659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873524    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-oauth-config\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.873659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873578    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-console-config\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.873659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873605    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-oauth-serving-cert\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.873659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873625    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg887\" (UniqueName: \"kubernetes.io/projected/34e97e2b-7229-4311-9657-4e5c09043921-kube-api-access-fg887\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.873659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873651    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-serving-cert\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.873924 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.873710    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-service-ca\") pod \"34e97e2b-7229-4311-9657-4e5c09043921\" (UID: \"34e97e2b-7229-4311-9657-4e5c09043921\") "
Jul 23 13:25:02.874052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.874007    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-console-config" (OuterVolumeSpecName: "console-config") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:25:02.874052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.874022    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:25:02.874052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.874002    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:25:02.874304 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.874226    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-service-ca" (OuterVolumeSpecName: "service-ca") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:25:02.875997 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.875979    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:25:02.876346 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.876324    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e97e2b-7229-4311-9657-4e5c09043921-kube-api-access-fg887" (OuterVolumeSpecName: "kube-api-access-fg887") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "kube-api-access-fg887". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:25:02.876385 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.876336    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "34e97e2b-7229-4311-9657-4e5c09043921" (UID: "34e97e2b-7229-4311-9657-4e5c09043921"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:25:02.956059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.956025    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv" event={"ID":"152956f6-7878-40e9-ac9e-405dc84d1eec","Type":"ContainerStarted","Data":"d6eba951ca84e676f740d1dbea8f58b5130e48b7bee78041be85f84e05faab5b"}
Jul 23 13:25:02.957243 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.957218    2559 generic.go:356] "Generic (PLEG): container finished" podID="8539726f-a4c9-41ed-bf95-202f1c1ec016" containerID="b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69" exitCode=0
Jul 23 13:25:02.957302 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.957259    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:25:02.957347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.957303    2559 scope.go:117] "RemoveContainer" containerID="b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69"
Jul 23 13:25:02.958375 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:02.958340    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" is forbidden: User \"system:node:ip-10-0-136-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-232.ec2.internal' and this object" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:25:02.958957 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.958934    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7" event={"ID":"e95dccf5-4c7d-4304-871d-ced1afe59c2f","Type":"ContainerStarted","Data":"a974a378492be125eaab0108d9c78a13cae52328cf6ad10efc383aa5c871e482"}
Jul 23 13:25:02.959072 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.959016    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:25:02.959072 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.959042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7" event={"ID":"e95dccf5-4c7d-4304-871d-ced1afe59c2f","Type":"ContainerStarted","Data":"e82e48318e7e3bc0134a99952e1bb207bba1526ad4be1e21aa1afb768e0c49d3"}
Jul 23 13:25:02.960279 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.960257    2559 generic.go:356] "Generic (PLEG): container finished" podID="34e97e2b-7229-4311-9657-4e5c09043921" containerID="ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401" exitCode=0
Jul 23 13:25:02.960387 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.960287    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87f4d49b-4dcls"
Jul 23 13:25:02.960387 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.960342    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87f4d49b-4dcls" event={"ID":"34e97e2b-7229-4311-9657-4e5c09043921","Type":"ContainerDied","Data":"ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401"}
Jul 23 13:25:02.960387 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.960376    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87f4d49b-4dcls" event={"ID":"34e97e2b-7229-4311-9657-4e5c09043921","Type":"ContainerDied","Data":"31c0432bbfeadd84acbd9f6ccb7070d3a8f7949808a8c506ac0c3c5664193b73"}
Jul 23 13:25:02.966173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.966157    2559 scope.go:117] "RemoveContainer" containerID="b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69"
Jul 23 13:25:02.966466 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:02.966431    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69\": container with ID starting with b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69 not found: ID does not exist" containerID="b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69"
Jul 23 13:25:02.966589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.966473    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69"} err="failed to get container status \"b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69\": rpc error: code = NotFound desc = could not find container \"b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69\": container with ID starting with b9b6e235de116a039dd4eb32aafba0245cfb172a2cbefc978c950cd159d8dc69 not found: ID does not exist"
Jul 23 13:25:02.966589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.966490    2559 scope.go:117] "RemoveContainer" containerID="ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401"
Jul 23 13:25:02.974025 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.973982    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wr9kv" podStartSLOduration=2.044882681 podStartE2EDuration="22.973966366s" totalImagesPullingTime="20.929083685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:25:02.973590358 +0000 UTC m=+433.349541136" watchObservedRunningTime="2026-07-23 13:25:02.973966366 +0000 UTC m=+433.349917144"
Jul 23 13:25:02.974757 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974741    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nqpt\" (UniqueName: \"kubernetes.io/projected/8539726f-a4c9-41ed-bf95-202f1c1ec016-kube-api-access-2nqpt\") pod \"8539726f-a4c9-41ed-bf95-202f1c1ec016\" (UID: \"8539726f-a4c9-41ed-bf95-202f1c1ec016\") "
Jul 23 13:25:02.974916 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974905    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-trusted-ca-bundle\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.974957 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974919    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-oauth-config\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.974957 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974929    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-console-config\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.974957 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974938    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-oauth-serving-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.974957 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974946    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg887\" (UniqueName: \"kubernetes.io/projected/34e97e2b-7229-4311-9657-4e5c09043921-kube-api-access-fg887\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.974957 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974954    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34e97e2b-7229-4311-9657-4e5c09043921-console-serving-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.975108 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.974962    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34e97e2b-7229-4311-9657-4e5c09043921-service-ca\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:02.975990 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.975968    2559 scope.go:117] "RemoveContainer" containerID="ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401"
Jul 23 13:25:02.976245 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:02.976225    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401\": container with ID starting with ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401 not found: ID does not exist" containerID="ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401"
Jul 23 13:25:02.976308 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.976251    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401"} err="failed to get container status \"ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401\": rpc error: code = NotFound desc = could not find container \"ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401\": container with ID starting with ed0129a6ccea60c7090ddd9d0f35768f4ce363f1e9b6e369a3d5f9901380b401 not found: ID does not exist"
Jul 23 13:25:02.985900 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.985864    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8539726f-a4c9-41ed-bf95-202f1c1ec016-kube-api-access-2nqpt" (OuterVolumeSpecName: "kube-api-access-2nqpt") pod "8539726f-a4c9-41ed-bf95-202f1c1ec016" (UID: "8539726f-a4c9-41ed-bf95-202f1c1ec016"). InnerVolumeSpecName "kube-api-access-2nqpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:25:02.988461 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.988440    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f87f4d49b-4dcls"]
Jul 23 13:25:02.996487 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:02.996459    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f87f4d49b-4dcls"]
Jul 23 13:25:02.996624 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:02.996603    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" is forbidden: User \"system:node:ip-10-0-136-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-232.ec2.internal' and this object" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:25:03.015284 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:03.015226    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7" podStartSLOduration=12.015205938 podStartE2EDuration="12.015205938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:25:03.013520825 +0000 UTC m=+433.389471603" watchObservedRunningTime="2026-07-23 13:25:03.015205938 +0000 UTC m=+433.391156718"
Jul 23 13:25:03.076220 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:03.076184    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nqpt\" (UniqueName: \"kubernetes.io/projected/8539726f-a4c9-41ed-bf95-202f1c1ec016-kube-api-access-2nqpt\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:03.269464 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:03.269431    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m6xhd\" is forbidden: User \"system:node:ip-10-0-136-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-232.ec2.internal' and this object" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m6xhd"
Jul 23 13:25:04.091414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:04.091380    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e97e2b-7229-4311-9657-4e5c09043921" path="/var/lib/kubelet/pods/34e97e2b-7229-4311-9657-4e5c09043921/volumes"
Jul 23 13:25:04.091794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:04.091708    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8539726f-a4c9-41ed-bf95-202f1c1ec016" path="/var/lib/kubelet/pods/8539726f-a4c9-41ed-bf95-202f1c1ec016/volumes"
Jul 23 13:25:08.152583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:08.152550    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:08.226370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:08.226341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:13.969454 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:13.969383    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjs7"
Jul 23 13:25:20.002877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.002833    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"]
Jul 23 13:25:20.003265 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.003198    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e97e2b-7229-4311-9657-4e5c09043921" containerName="console"
Jul 23 13:25:20.003265 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.003212    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e97e2b-7229-4311-9657-4e5c09043921" containerName="console"
Jul 23 13:25:20.003352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.003284    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e97e2b-7229-4311-9657-4e5c09043921" containerName="console"
Jul 23 13:25:20.019024 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.018983    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"]
Jul 23 13:25:20.019202 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.019111    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.021861 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.021837    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gsb68\""
Jul 23 13:25:20.035068 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.035037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:20.122474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122522    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122644    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtcdp\" (UniqueName: \"kubernetes.io/projected/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-kube-api-access-xtcdp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122669    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.122880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.122812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224197 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224157    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224197 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224340    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224678 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224626    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtcdp\" (UniqueName: \"kubernetes.io/projected/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-kube-api-access-xtcdp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224736 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224672    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224736 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224637    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.224736 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224708    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.225206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.224971    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.225206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.225004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.225206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.225178    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.225412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.225220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.227024 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.227004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.227412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.227387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.235038 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.234992    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.235038 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.235010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtcdp\" (UniqueName: \"kubernetes.io/projected/35b6cbcf-290d-4c3b-81ee-92b5c7b6f852-kube-api-access-xtcdp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bn594\" (UID: \"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.332244 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.332221    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:20.365747 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.365712    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:20.508552 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.508506    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"]
Jul 23 13:25:20.510727 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:25:20.510701    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b6cbcf_290d_4c3b_81ee_92b5c7b6f852.slice/crio-df06da329f1b6d6b8ee46f4acfeefb4076b4533bdd89a663da5fdc9ca9a13cb7 WatchSource:0}: Error finding container df06da329f1b6d6b8ee46f4acfeefb4076b4533bdd89a663da5fdc9ca9a13cb7: Status 404 returned error can't find the container with id df06da329f1b6d6b8ee46f4acfeefb4076b4533bdd89a663da5fdc9ca9a13cb7
Jul 23 13:25:20.513209 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.513180    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:25:20.513301 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.513253    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:25:20.513360 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:20.513296    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:25:21.045445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:21.045358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" event={"ID":"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852","Type":"ContainerStarted","Data":"8ab94aa97446fa5faab7d50e33d6a2a9b4c0ef688416945070694726b320f36e"}
Jul 23 13:25:21.045838 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:21.045454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" event={"ID":"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852","Type":"ContainerStarted","Data":"df06da329f1b6d6b8ee46f4acfeefb4076b4533bdd89a663da5fdc9ca9a13cb7"}
Jul 23 13:25:21.067946 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:21.067897    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" podStartSLOduration=2.067880302 podStartE2EDuration="2.067880302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:25:21.06646567 +0000 UTC m=+451.442416450" watchObservedRunningTime="2026-07-23 13:25:21.067880302 +0000 UTC m=+451.443831079"
Jul 23 13:25:21.332775 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:21.332741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:21.337760 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:21.337739    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:22.050200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:22.050168    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:22.051038 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:22.051021    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:25:35.433049 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.433011    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:35.435027 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.434998    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:35.525110 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.525068    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:35.528127 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.528100    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:35.528227 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.528146    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:35.528267 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.528233    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.531097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.531072    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 13:25:35.552789 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.552761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4zc2\" (UniqueName: \"kubernetes.io/projected/32d22157-29c5-4853-898d-7dbc38d1493c-kube-api-access-w4zc2\") pod \"limitador-limitador-64c94fd64f-lj5g7\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.552947 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.552930    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d22157-29c5-4853-898d-7dbc38d1493c-config-file\") pod \"limitador-limitador-64c94fd64f-lj5g7\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.624963 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.624930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:35.654005 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.653968    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d22157-29c5-4853-898d-7dbc38d1493c-config-file\") pod \"limitador-limitador-64c94fd64f-lj5g7\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.654172 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.654026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4zc2\" (UniqueName: \"kubernetes.io/projected/32d22157-29c5-4853-898d-7dbc38d1493c-kube-api-access-w4zc2\") pod \"limitador-limitador-64c94fd64f-lj5g7\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.654640 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.654623    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d22157-29c5-4853-898d-7dbc38d1493c-config-file\") pod \"limitador-limitador-64c94fd64f-lj5g7\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.662551 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.662510    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4zc2\" (UniqueName: \"kubernetes.io/projected/32d22157-29c5-4853-898d-7dbc38d1493c-kube-api-access-w4zc2\") pod \"limitador-limitador-64c94fd64f-lj5g7\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:35.839776 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:35.839741    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:36.000257 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:36.000231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:36.002312 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:25:36.002283    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d22157_29c5_4853_898d_7dbc38d1493c.slice/crio-12593d424473718ee554f57fc8312df1e8a00953238c484398f57abebdb4f840 WatchSource:0}: Error finding container 12593d424473718ee554f57fc8312df1e8a00953238c484398f57abebdb4f840: Status 404 returned error can't find the container with id 12593d424473718ee554f57fc8312df1e8a00953238c484398f57abebdb4f840
Jul 23 13:25:36.115506 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:36.115428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7" event={"ID":"32d22157-29c5-4853-898d-7dbc38d1493c","Type":"ContainerStarted","Data":"12593d424473718ee554f57fc8312df1e8a00953238c484398f57abebdb4f840"}
Jul 23 13:25:36.281366 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:36.281331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:36.365474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:36.365437    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:39.134526 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:39.134441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7" event={"ID":"32d22157-29c5-4853-898d-7dbc38d1493c","Type":"ContainerStarted","Data":"35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af"}
Jul 23 13:25:39.134937 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:39.134566    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:39.150396 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:39.150356    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7" podStartSLOduration=1.313899768 podStartE2EDuration="4.150340839s" totalImagesPullingTime="2.836441071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:25:39.149501755 +0000 UTC m=+469.525452532" watchObservedRunningTime="2026-07-23 13:25:39.150340839 +0000 UTC m=+469.526291618"
Jul 23 13:25:50.140001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:50.139973    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:53.445402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:53.445369    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:53.445882 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:53.445579    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7" podUID="32d22157-29c5-4853-898d-7dbc38d1493c" containerName="limitador" containerID="cri-o://35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af" gracePeriod=30
Jul 23 13:25:53.999273 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:53.999250    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:54.117707 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.117651    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4zc2\" (UniqueName: \"kubernetes.io/projected/32d22157-29c5-4853-898d-7dbc38d1493c-kube-api-access-w4zc2\") pod \"32d22157-29c5-4853-898d-7dbc38d1493c\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") "
Jul 23 13:25:54.117707 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.117721    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d22157-29c5-4853-898d-7dbc38d1493c-config-file\") pod \"32d22157-29c5-4853-898d-7dbc38d1493c\" (UID: \"32d22157-29c5-4853-898d-7dbc38d1493c\") "
Jul 23 13:25:54.118112 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.118076    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32d22157-29c5-4853-898d-7dbc38d1493c-config-file" (OuterVolumeSpecName: "config-file") pod "32d22157-29c5-4853-898d-7dbc38d1493c" (UID: "32d22157-29c5-4853-898d-7dbc38d1493c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:25:54.119949 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.119921    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d22157-29c5-4853-898d-7dbc38d1493c-kube-api-access-w4zc2" (OuterVolumeSpecName: "kube-api-access-w4zc2") pod "32d22157-29c5-4853-898d-7dbc38d1493c" (UID: "32d22157-29c5-4853-898d-7dbc38d1493c"). InnerVolumeSpecName "kube-api-access-w4zc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:25:54.201784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.201749    2559 generic.go:356] "Generic (PLEG): container finished" podID="32d22157-29c5-4853-898d-7dbc38d1493c" containerID="35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af" exitCode=0
Jul 23 13:25:54.201784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.201786    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7" event={"ID":"32d22157-29c5-4853-898d-7dbc38d1493c","Type":"ContainerDied","Data":"35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af"}
Jul 23 13:25:54.201991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.201802    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"
Jul 23 13:25:54.201991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.201823    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lj5g7" event={"ID":"32d22157-29c5-4853-898d-7dbc38d1493c","Type":"ContainerDied","Data":"12593d424473718ee554f57fc8312df1e8a00953238c484398f57abebdb4f840"}
Jul 23 13:25:54.201991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.201844    2559 scope.go:117] "RemoveContainer" containerID="35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af"
Jul 23 13:25:54.210810 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.210788    2559 scope.go:117] "RemoveContainer" containerID="35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af"
Jul 23 13:25:54.211079 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:25:54.211058    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af\": container with ID starting with 35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af not found: ID does not exist" containerID="35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af"
Jul 23 13:25:54.211143 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.211093    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af"} err="failed to get container status \"35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af\": rpc error: code = NotFound desc = could not find container \"35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af\": container with ID starting with 35599c1cc63e88bed54422345438e96619cd44c06ecf10a530c04ce16cc620af not found: ID does not exist"
Jul 23 13:25:54.219327 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.219304    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d22157-29c5-4853-898d-7dbc38d1493c-config-file\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:54.219327 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.219328    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4zc2\" (UniqueName: \"kubernetes.io/projected/32d22157-29c5-4853-898d-7dbc38d1493c-kube-api-access-w4zc2\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:25:54.222493 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.222458    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:54.226473 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:54.226451    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lj5g7"]
Jul 23 13:25:56.097320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:56.097288    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32d22157-29c5-4853-898d-7dbc38d1493c" path="/var/lib/kubelet/pods/32d22157-29c5-4853-898d-7dbc38d1493c/volumes"
Jul 23 13:25:58.823467 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:58.823435    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:58.889968 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:58.889935    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:59.263604 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.263575    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"]
Jul 23 13:25:59.263997 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.263972    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32d22157-29c5-4853-898d-7dbc38d1493c" containerName="limitador"
Jul 23 13:25:59.263997 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.263995    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="32d22157-29c5-4853-898d-7dbc38d1493c" containerName="limitador"
Jul 23 13:25:59.264203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.264064    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="32d22157-29c5-4853-898d-7dbc38d1493c" containerName="limitador"
Jul 23 13:25:59.268361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.268339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.271292 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.271268    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 13:25:59.272171 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.272153    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-flflk\""
Jul 23 13:25:59.272171 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.272159    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 13:25:59.272332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.272144    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 23 13:25:59.279240 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.279217    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"]
Jul 23 13:25:59.302827 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.302791    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:25:59.362332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.362308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5jg4\" (UniqueName: \"kubernetes.io/projected/29ef5b5b-3e81-4787-885f-6adb17fe362f-kube-api-access-f5jg4\") pod \"postgres-69fb4594cf-fvqtf\" (UID: \"29ef5b5b-3e81-4787-885f-6adb17fe362f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.362451 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.362342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29ef5b5b-3e81-4787-885f-6adb17fe362f-data\") pod \"postgres-69fb4594cf-fvqtf\" (UID: \"29ef5b5b-3e81-4787-885f-6adb17fe362f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.463054 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.463026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29ef5b5b-3e81-4787-885f-6adb17fe362f-data\") pod \"postgres-69fb4594cf-fvqtf\" (UID: \"29ef5b5b-3e81-4787-885f-6adb17fe362f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.463216 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.463118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5jg4\" (UniqueName: \"kubernetes.io/projected/29ef5b5b-3e81-4787-885f-6adb17fe362f-kube-api-access-f5jg4\") pod \"postgres-69fb4594cf-fvqtf\" (UID: \"29ef5b5b-3e81-4787-885f-6adb17fe362f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.463422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.463404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29ef5b5b-3e81-4787-885f-6adb17fe362f-data\") pod \"postgres-69fb4594cf-fvqtf\" (UID: \"29ef5b5b-3e81-4787-885f-6adb17fe362f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.475498 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.475475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5jg4\" (UniqueName: \"kubernetes.io/projected/29ef5b5b-3e81-4787-885f-6adb17fe362f-kube-api-access-f5jg4\") pod \"postgres-69fb4594cf-fvqtf\" (UID: \"29ef5b5b-3e81-4787-885f-6adb17fe362f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.580673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.580637    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:25:59.709973 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:25:59.709951    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"]
Jul 23 13:25:59.712269 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:25:59.712243    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ef5b5b_3e81_4787_885f_6adb17fe362f.slice/crio-55e60076cef14a24de9b12abe06e24f18b9a073e6e76c4ba38f6375d5eecee48 WatchSource:0}: Error finding container 55e60076cef14a24de9b12abe06e24f18b9a073e6e76c4ba38f6375d5eecee48: Status 404 returned error can't find the container with id 55e60076cef14a24de9b12abe06e24f18b9a073e6e76c4ba38f6375d5eecee48
Jul 23 13:26:00.230168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:00.230135    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf" event={"ID":"29ef5b5b-3e81-4787-885f-6adb17fe362f","Type":"ContainerStarted","Data":"55e60076cef14a24de9b12abe06e24f18b9a073e6e76c4ba38f6375d5eecee48"}
Jul 23 13:26:05.256427 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:05.256393    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf" event={"ID":"29ef5b5b-3e81-4787-885f-6adb17fe362f","Type":"ContainerStarted","Data":"0a95b506d08c42d844abbc748135c6a4fb426dff00fb0b751e1cffc350d01fc2"}
Jul 23 13:26:05.256839 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:05.256510    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:26:05.289473 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:05.289419    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf" podStartSLOduration=1.2161847510000001 podStartE2EDuration="6.28940493s" totalImagesPullingTime="5.073220179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:26:05.287951243 +0000 UTC m=+495.663902021" watchObservedRunningTime="2026-07-23 13:26:05.28940493 +0000 UTC m=+495.665355707"
Jul 23 13:26:08.860799 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:08.860757    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:08.946462 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:08.946428    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:11.291252 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:11.291219    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fvqtf"
Jul 23 13:26:11.665739 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:11.665702    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:12.077184 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:12.077105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:18.704484 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.704447    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75d54c77c7-d9bws"]
Jul 23 13:26:18.712974 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.712942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:18.721802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.721780    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75d54c77c7-d9bws"]
Jul 23 13:26:18.721925 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.721873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:18.725074 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.725053    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h7qbw\""
Jul 23 13:26:18.725202 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.725056    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 13:26:18.725982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.725964    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 13:26:18.836932 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.836894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27h5t\" (UniqueName: \"kubernetes.io/projected/ab6383c9-e650-4b35-8b93-02d05d7815d4-kube-api-access-27h5t\") pod \"maas-controller-75d54c77c7-d9bws\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") " pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:18.837095 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.836956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab6383c9-e650-4b35-8b93-02d05d7815d4-webhook-cert\") pod \"maas-controller-75d54c77c7-d9bws\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") " pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:18.844294 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.844270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:18.910064 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.910039    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-8krpd"]
Jul 23 13:26:18.912863 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.912845    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:18.927029 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.927002    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-8krpd"]
Jul 23 13:26:18.931756 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.931729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:18.937568 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.937545    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab6383c9-e650-4b35-8b93-02d05d7815d4-webhook-cert\") pod \"maas-controller-75d54c77c7-d9bws\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") " pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:18.937693 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.937669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27h5t\" (UniqueName: \"kubernetes.io/projected/ab6383c9-e650-4b35-8b93-02d05d7815d4-kube-api-access-27h5t\") pod \"maas-controller-75d54c77c7-d9bws\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") " pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:18.940252 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.940233    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab6383c9-e650-4b35-8b93-02d05d7815d4-webhook-cert\") pod \"maas-controller-75d54c77c7-d9bws\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") " pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:18.947234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:18.947212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27h5t\" (UniqueName: \"kubernetes.io/projected/ab6383c9-e650-4b35-8b93-02d05d7815d4-kube-api-access-27h5t\") pod \"maas-controller-75d54c77c7-d9bws\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") " pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:19.033351 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.033278    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:19.038724 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.038697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdbdj\" (UniqueName: \"kubernetes.io/projected/e982661e-f60b-4b28-b132-067c346fa4d3-kube-api-access-xdbdj\") pod \"maas-controller-7fdc9c77b7-8krpd\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") " pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.038845 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.038785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e982661e-f60b-4b28-b132-067c346fa4d3-webhook-cert\") pod \"maas-controller-7fdc9c77b7-8krpd\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") " pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.139298 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.139241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdbdj\" (UniqueName: \"kubernetes.io/projected/e982661e-f60b-4b28-b132-067c346fa4d3-kube-api-access-xdbdj\") pod \"maas-controller-7fdc9c77b7-8krpd\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") " pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.139477 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.139369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e982661e-f60b-4b28-b132-067c346fa4d3-webhook-cert\") pod \"maas-controller-7fdc9c77b7-8krpd\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") " pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.142024 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.141996    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e982661e-f60b-4b28-b132-067c346fa4d3-webhook-cert\") pod \"maas-controller-7fdc9c77b7-8krpd\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") " pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.149527 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.149503    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdbdj\" (UniqueName: \"kubernetes.io/projected/e982661e-f60b-4b28-b132-067c346fa4d3-kube-api-access-xdbdj\") pod \"maas-controller-7fdc9c77b7-8krpd\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") " pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.167055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.167034    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75d54c77c7-d9bws"]
Jul 23 13:26:19.168286 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:26:19.168262    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6383c9_e650_4b35_8b93_02d05d7815d4.slice/crio-1571bd113d9b8716a465b4508871e6fc2e350b138a4b9a17e014486a2cf8eaf8 WatchSource:0}: Error finding container 1571bd113d9b8716a465b4508871e6fc2e350b138a4b9a17e014486a2cf8eaf8: Status 404 returned error can't find the container with id 1571bd113d9b8716a465b4508871e6fc2e350b138a4b9a17e014486a2cf8eaf8
Jul 23 13:26:19.222678 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.222657    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:19.319966 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.319938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d54c77c7-d9bws" event={"ID":"ab6383c9-e650-4b35-8b93-02d05d7815d4","Type":"ContainerStarted","Data":"1571bd113d9b8716a465b4508871e6fc2e350b138a4b9a17e014486a2cf8eaf8"}
Jul 23 13:26:19.357706 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:19.357683    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-8krpd"]
Jul 23 13:26:19.359670 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:26:19.359646    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode982661e_f60b_4b28_b132_067c346fa4d3.slice/crio-164fb93be4c0c5a13e60f46ee8c81532393f2112bab95c7d1073f89fba7c3373 WatchSource:0}: Error finding container 164fb93be4c0c5a13e60f46ee8c81532393f2112bab95c7d1073f89fba7c3373: Status 404 returned error can't find the container with id 164fb93be4c0c5a13e60f46ee8c81532393f2112bab95c7d1073f89fba7c3373
Jul 23 13:26:20.333689 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:20.333608    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd" event={"ID":"e982661e-f60b-4b28-b132-067c346fa4d3","Type":"ContainerStarted","Data":"164fb93be4c0c5a13e60f46ee8c81532393f2112bab95c7d1073f89fba7c3373"}
Jul 23 13:26:22.047003 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.046929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:22.148055 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.147963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:22.344877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.344838    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d54c77c7-d9bws" event={"ID":"ab6383c9-e650-4b35-8b93-02d05d7815d4","Type":"ContainerStarted","Data":"f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa"}
Jul 23 13:26:22.345056 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.345032    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:22.346247 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.346220    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd" event={"ID":"e982661e-f60b-4b28-b132-067c346fa4d3","Type":"ContainerStarted","Data":"450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335"}
Jul 23 13:26:22.346381 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.346340    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:22.366517 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:22.366466    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75d54c77c7-d9bws" podStartSLOduration=1.6566843900000001 podStartE2EDuration="4.366450762s" totalImagesPullingTime="2.709766372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:26:22.36518858 +0000 UTC m=+512.741139359" watchObservedRunningTime="2026-07-23 13:26:22.366450762 +0000 UTC m=+512.742401541"
Jul 23 13:26:33.353250 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.353218    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:33.353658 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.353294    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:26:33.378746 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.378694    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd" podStartSLOduration=12.8670459 podStartE2EDuration="15.378677717s" totalImagesPullingTime="2.511631817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:26:22.387134454 +0000 UTC m=+512.763085232" watchObservedRunningTime="2026-07-23 13:26:33.378677717 +0000 UTC m=+523.754628494"
Jul 23 13:26:33.428382 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.428348    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75d54c77c7-d9bws"]
Jul 23 13:26:33.428852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.428813    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75d54c77c7-d9bws" podUID="ab6383c9-e650-4b35-8b93-02d05d7815d4" containerName="manager" containerID="cri-o://f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa" gracePeriod=30
Jul 23 13:26:33.683410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.683381    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:33.766892 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.766862    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab6383c9-e650-4b35-8b93-02d05d7815d4-webhook-cert\") pod \"ab6383c9-e650-4b35-8b93-02d05d7815d4\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") "
Jul 23 13:26:33.767061 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.766947    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27h5t\" (UniqueName: \"kubernetes.io/projected/ab6383c9-e650-4b35-8b93-02d05d7815d4-kube-api-access-27h5t\") pod \"ab6383c9-e650-4b35-8b93-02d05d7815d4\" (UID: \"ab6383c9-e650-4b35-8b93-02d05d7815d4\") "
Jul 23 13:26:33.769244 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.769216    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab6383c9-e650-4b35-8b93-02d05d7815d4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ab6383c9-e650-4b35-8b93-02d05d7815d4" (UID: "ab6383c9-e650-4b35-8b93-02d05d7815d4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:26:33.769347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.769315    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6383c9-e650-4b35-8b93-02d05d7815d4-kube-api-access-27h5t" (OuterVolumeSpecName: "kube-api-access-27h5t") pod "ab6383c9-e650-4b35-8b93-02d05d7815d4" (UID: "ab6383c9-e650-4b35-8b93-02d05d7815d4"). InnerVolumeSpecName "kube-api-access-27h5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:26:33.867777 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.867742    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27h5t\" (UniqueName: \"kubernetes.io/projected/ab6383c9-e650-4b35-8b93-02d05d7815d4-kube-api-access-27h5t\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:26:33.867777 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:33.867774    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab6383c9-e650-4b35-8b93-02d05d7815d4-webhook-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:26:34.401043 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.400953    2559 generic.go:356] "Generic (PLEG): container finished" podID="ab6383c9-e650-4b35-8b93-02d05d7815d4" containerID="f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa" exitCode=0
Jul 23 13:26:34.401043 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.400997    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d54c77c7-d9bws"
Jul 23 13:26:34.401043 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.401009    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d54c77c7-d9bws" event={"ID":"ab6383c9-e650-4b35-8b93-02d05d7815d4","Type":"ContainerDied","Data":"f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa"}
Jul 23 13:26:34.401584 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.401063    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d54c77c7-d9bws" event={"ID":"ab6383c9-e650-4b35-8b93-02d05d7815d4","Type":"ContainerDied","Data":"1571bd113d9b8716a465b4508871e6fc2e350b138a4b9a17e014486a2cf8eaf8"}
Jul 23 13:26:34.401584 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.401091    2559 scope.go:117] "RemoveContainer" containerID="f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa"
Jul 23 13:26:34.409442 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.409425    2559 scope.go:117] "RemoveContainer" containerID="f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa"
Jul 23 13:26:34.409726 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:26:34.409698    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa\": container with ID starting with f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa not found: ID does not exist" containerID="f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa"
Jul 23 13:26:34.409796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.409731    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa"} err="failed to get container status \"f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa\": rpc error: code = NotFound desc = could not find container \"f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa\": container with ID starting with f4041028f5a9f09dbbdc6505ce12e455324e928f84f70368f4e55af209a63aaa not found: ID does not exist"
Jul 23 13:26:34.419553 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.419516    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75d54c77c7-d9bws"]
Jul 23 13:26:34.427018 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:34.426998    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75d54c77c7-d9bws"]
Jul 23 13:26:36.091545 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:36.091487    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab6383c9-e650-4b35-8b93-02d05d7815d4" path="/var/lib/kubelet/pods/ab6383c9-e650-4b35-8b93-02d05d7815d4/volumes"
Jul 23 13:26:49.625230 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:49.625151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:54.333070 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:54.333033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:54.435119 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:54.435088    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:54.881075 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:54.881043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:54.983317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:54.983282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:55.404874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.404837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:55.508926 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.508896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:26:55.518149 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.518121    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"]
Jul 23 13:26:55.518526 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.518482    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab6383c9-e650-4b35-8b93-02d05d7815d4" containerName="manager"
Jul 23 13:26:55.518526 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.518500    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6383c9-e650-4b35-8b93-02d05d7815d4" containerName="manager"
Jul 23 13:26:55.518799 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.518601    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab6383c9-e650-4b35-8b93-02d05d7815d4" containerName="manager"
Jul 23 13:26:55.527576 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.527558    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.534829 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.534805    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 23 13:26:55.535706 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.535267    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"]
Jul 23 13:26:55.559969 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.559937    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-cd4pt"]
Jul 23 13:26:55.564670 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.564642    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.567746 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.567723    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w54nd\""
Jul 23 13:26:55.576566 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.576526    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-cd4pt"]
Jul 23 13:26:55.656394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.656303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnr9z\" (UniqueName: \"kubernetes.io/projected/bf34866c-a975-401c-8bdb-2dbd62682f76-kube-api-access-lnr9z\") pod \"payload-pre-processing-7677b774bb-mmvwx\" (UID: \"bf34866c-a975-401c-8bdb-2dbd62682f76\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.656394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.656349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf34866c-a975-401c-8bdb-2dbd62682f76-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mmvwx\" (UID: \"bf34866c-a975-401c-8bdb-2dbd62682f76\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.656687 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.656558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702fe9b2-d746-4577-9caa-120bae5e0518-plugins-config-volume\") pod \"payload-processing-5bf47cd988-cd4pt\" (UID: \"702fe9b2-d746-4577-9caa-120bae5e0518\") " pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.656687 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.656614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4xh2\" (UniqueName: \"kubernetes.io/projected/702fe9b2-d746-4577-9caa-120bae5e0518-kube-api-access-g4xh2\") pod \"payload-processing-5bf47cd988-cd4pt\" (UID: \"702fe9b2-d746-4577-9caa-120bae5e0518\") " pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.757996 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.757956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702fe9b2-d746-4577-9caa-120bae5e0518-plugins-config-volume\") pod \"payload-processing-5bf47cd988-cd4pt\" (UID: \"702fe9b2-d746-4577-9caa-120bae5e0518\") " pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.758210 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.758006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4xh2\" (UniqueName: \"kubernetes.io/projected/702fe9b2-d746-4577-9caa-120bae5e0518-kube-api-access-g4xh2\") pod \"payload-processing-5bf47cd988-cd4pt\" (UID: \"702fe9b2-d746-4577-9caa-120bae5e0518\") " pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.758210 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.758075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnr9z\" (UniqueName: \"kubernetes.io/projected/bf34866c-a975-401c-8bdb-2dbd62682f76-kube-api-access-lnr9z\") pod \"payload-pre-processing-7677b774bb-mmvwx\" (UID: \"bf34866c-a975-401c-8bdb-2dbd62682f76\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.758210 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.758104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf34866c-a975-401c-8bdb-2dbd62682f76-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mmvwx\" (UID: \"bf34866c-a975-401c-8bdb-2dbd62682f76\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.758995 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.758971    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702fe9b2-d746-4577-9caa-120bae5e0518-plugins-config-volume\") pod \"payload-processing-5bf47cd988-cd4pt\" (UID: \"702fe9b2-d746-4577-9caa-120bae5e0518\") " pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.759041 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.758971    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf34866c-a975-401c-8bdb-2dbd62682f76-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mmvwx\" (UID: \"bf34866c-a975-401c-8bdb-2dbd62682f76\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.770760 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.770721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4xh2\" (UniqueName: \"kubernetes.io/projected/702fe9b2-d746-4577-9caa-120bae5e0518-kube-api-access-g4xh2\") pod \"payload-processing-5bf47cd988-cd4pt\" (UID: \"702fe9b2-d746-4577-9caa-120bae5e0518\") " pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:55.770886 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.770763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnr9z\" (UniqueName: \"kubernetes.io/projected/bf34866c-a975-401c-8bdb-2dbd62682f76-kube-api-access-lnr9z\") pod \"payload-pre-processing-7677b774bb-mmvwx\" (UID: \"bf34866c-a975-401c-8bdb-2dbd62682f76\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.840577 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.840526    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:55.875739 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:55.875712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:56.033781 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:56.033754    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"]
Jul 23 13:26:56.035906 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:26:56.035873    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf34866c_a975_401c_8bdb_2dbd62682f76.slice/crio-1b0e2f24969571a2ea1a08646897011a8fc251e5cd2c4152c48daf8b39e3e199 WatchSource:0}: Error finding container 1b0e2f24969571a2ea1a08646897011a8fc251e5cd2c4152c48daf8b39e3e199: Status 404 returned error can't find the container with id 1b0e2f24969571a2ea1a08646897011a8fc251e5cd2c4152c48daf8b39e3e199
Jul 23 13:26:56.046794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:56.046771    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-cd4pt"]
Jul 23 13:26:56.048861 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:26:56.048837    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702fe9b2_d746_4577_9caa_120bae5e0518.slice/crio-5d4a753f85999becbc76695c17b46bd51a15640ab1c07ba15124539d4a31ab1c WatchSource:0}: Error finding container 5d4a753f85999becbc76695c17b46bd51a15640ab1c07ba15124539d4a31ab1c: Status 404 returned error can't find the container with id 5d4a753f85999becbc76695c17b46bd51a15640ab1c07ba15124539d4a31ab1c
Jul 23 13:26:56.508575 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:56.507389    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt" event={"ID":"702fe9b2-d746-4577-9caa-120bae5e0518","Type":"ContainerStarted","Data":"5d4a753f85999becbc76695c17b46bd51a15640ab1c07ba15124539d4a31ab1c"}
Jul 23 13:26:56.510321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:56.510288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx" event={"ID":"bf34866c-a975-401c-8bdb-2dbd62682f76","Type":"ContainerStarted","Data":"1b0e2f24969571a2ea1a08646897011a8fc251e5cd2c4152c48daf8b39e3e199"}
Jul 23 13:26:59.529186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:59.529024    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt" event={"ID":"702fe9b2-d746-4577-9caa-120bae5e0518","Type":"ContainerStarted","Data":"d654afcf9931352a215761672661bd2df9282207ffc0961deb4a07281975a936"}
Jul 23 13:26:59.529186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:59.529103    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:26:59.530405 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:59.530369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx" event={"ID":"bf34866c-a975-401c-8bdb-2dbd62682f76","Type":"ContainerStarted","Data":"05d66ac770b5b4abf467164473fa5ae4226bc632f575b7b93f0f1118d9b3f88d"}
Jul 23 13:26:59.530522 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:59.530496    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:26:59.549603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:59.549527    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt" podStartSLOduration=1.383190411 podStartE2EDuration="4.549516444s" totalImagesPullingTime="3.166326033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:26:59.547959162 +0000 UTC m=+549.923909939" watchObservedRunningTime="2026-07-23 13:26:59.549516444 +0000 UTC m=+549.925467248"
Jul 23 13:26:59.565967 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:26:59.565928    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx" podStartSLOduration=1.396442826 podStartE2EDuration="4.565910035s" totalImagesPullingTime="3.169467209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:26:59.564633303 +0000 UTC m=+549.940584078" watchObservedRunningTime="2026-07-23 13:26:59.565910035 +0000 UTC m=+549.941860814"
Jul 23 13:27:03.552478 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:03.552384    2559 generic.go:356] "Generic (PLEG): container finished" podID="35b6cbcf-290d-4c3b-81ee-92b5c7b6f852" containerID="8ab94aa97446fa5faab7d50e33d6a2a9b4c0ef688416945070694726b320f36e" exitCode=0
Jul 23 13:27:03.552478 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:03.552462    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" event={"ID":"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852","Type":"ContainerDied","Data":"8ab94aa97446fa5faab7d50e33d6a2a9b4c0ef688416945070694726b320f36e"}
Jul 23 13:27:03.552913 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:03.552899    2559 scope.go:117] "RemoveContainer" containerID="8ab94aa97446fa5faab7d50e33d6a2a9b4c0ef688416945070694726b320f36e"
Jul 23 13:27:03.554004 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:03.553979    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:27:03.554097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:03.554041    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:27:03.554097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:03.554073    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:27:04.558780 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:04.558742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" event={"ID":"35b6cbcf-290d-4c3b-81ee-92b5c7b6f852","Type":"ContainerStarted","Data":"754d0e47de8187e43dbbd25c58f6be84c240371ceacb54733f07f65bc83ea77b"}
Jul 23 13:27:05.250176 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:05.250128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:05.332684 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:05.332640    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:27:05.334045 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:05.334020    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bn594 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 23 13:27:05.334108 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:05.334078    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" podUID="35b6cbcf-290d-4c3b-81ee-92b5c7b6f852" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 23 13:27:05.352947 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:05.352911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:06.333186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:06.333152    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bn594 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 23 13:27:06.333602 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:06.333208    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" podUID="35b6cbcf-290d-4c3b-81ee-92b5c7b6f852" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 23 13:27:07.051119 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:07.051084    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:27:07.333588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:07.333557    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bn594 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 23 13:27:07.333966 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:07.333628    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" podUID="35b6cbcf-290d-4c3b-81ee-92b5c7b6f852" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 23 13:27:08.333548 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:08.333505    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bn594 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 23 13:27:08.333723 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:08.333593    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594" podUID="35b6cbcf-290d-4c3b-81ee-92b5c7b6f852" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 23 13:27:09.336232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:09.336199    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:27:09.337067 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:09.337051    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bn594"
Jul 23 13:27:10.537729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:10.537695    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-cd4pt"
Jul 23 13:27:10.538183 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:10.537869    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mmvwx"
Jul 23 13:27:11.261768 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.261731    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"]
Jul 23 13:27:11.265327 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.265304    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.268109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.268086    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 13:27:11.269231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.269204    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 13:27:11.269231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.269225    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 23 13:27:11.269363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.269275    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lvdfp\""
Jul 23 13:27:11.274871 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.274847    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"]
Jul 23 13:27:11.290097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.290062    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:11.317297 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.317269    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5r6n\" (UniqueName: \"kubernetes.io/projected/8f27af21-2fdc-4481-ad08-a980a3de12fd-kube-api-access-m5r6n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.317422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.317310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.317422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.317328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.317422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.317379    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.317603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.317441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.317603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.317473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f27af21-2fdc-4481-ad08-a980a3de12fd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.418374 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5r6n\" (UniqueName: \"kubernetes.io/projected/8f27af21-2fdc-4481-ad08-a980a3de12fd-kube-api-access-m5r6n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.418754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.418754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.418754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418458    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.418754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.418754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418569    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f27af21-2fdc-4481-ad08-a980a3de12fd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.419127 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.418997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.419127 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.419121    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.419332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.419302    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.421097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.421069    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f27af21-2fdc-4481-ad08-a980a3de12fd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.421097 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.421090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f27af21-2fdc-4481-ad08-a980a3de12fd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.430718 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.430698    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5r6n\" (UniqueName: \"kubernetes.io/projected/8f27af21-2fdc-4481-ad08-a980a3de12fd-kube-api-access-m5r6n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql\" (UID: \"8f27af21-2fdc-4481-ad08-a980a3de12fd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.577747 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.577678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:11.716590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:11.716513    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"]
Jul 23 13:27:11.718894 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:27:11.718868    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f27af21_2fdc_4481_ad08_a980a3de12fd.slice/crio-0d71ac2b28986229ab19c31c32feb3703c6cc8df7cc73b20e65e57f26e39a8f8 WatchSource:0}: Error finding container 0d71ac2b28986229ab19c31c32feb3703c6cc8df7cc73b20e65e57f26e39a8f8: Status 404 returned error can't find the container with id 0d71ac2b28986229ab19c31c32feb3703c6cc8df7cc73b20e65e57f26e39a8f8
Jul 23 13:27:12.598698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:12.598665    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql" event={"ID":"8f27af21-2fdc-4481-ad08-a980a3de12fd","Type":"ContainerStarted","Data":"0d71ac2b28986229ab19c31c32feb3703c6cc8df7cc73b20e65e57f26e39a8f8"}
Jul 23 13:27:15.622317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:15.622279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql" event={"ID":"8f27af21-2fdc-4481-ad08-a980a3de12fd","Type":"ContainerStarted","Data":"0633397cd3c247310c7e83b0699c820d8515b48f446f0e90f34cff8ef9d78c2d"}
Jul 23 13:27:15.643887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:15.643838    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql" podStartSLOduration=1.192983426 podStartE2EDuration="4.643822159s" totalImagesPullingTime="3.450838733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:27:15.642052251 +0000 UTC m=+566.018003030" watchObservedRunningTime="2026-07-23 13:27:15.643822159 +0000 UTC m=+566.019772937"
Jul 23 13:27:21.578467 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:21.578435    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:21.579051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:21.578488    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:21.593991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:21.593967    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:21.660248 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:21.660224    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql"
Jul 23 13:27:21.987025 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:21.986935    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:22.086378 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:22.086346    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:26.797384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:26.797349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:45.119775 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:45.119740    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:27:53.464027 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:27:53.463979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:28:00.483592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:28:00.483558    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:28:13.432790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:28:13.432700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:28:35.056269 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:28:35.056227    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:28:35.156001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:28:35.155970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:00.014939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:00.014905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:00.081801 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:00.081754    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:04.752679 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:04.752622    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:15.144683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:15.144653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:15.240309 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:15.240276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:16.543994 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:16.543955    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:29:16.666869 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:29:16.666834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:30:00.167194 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.167115    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"]
Jul 23 13:30:00.170654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.170636    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"
Jul 23 13:30:00.174330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.174310    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnln8\""
Jul 23 13:30:00.178228 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.178202    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"]
Jul 23 13:30:00.181516 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.181495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"
Jul 23 13:30:00.331296 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.331157    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"]
Jul 23 13:30:00.333708 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:30:00.333675    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41c1db3_127a_4fcb_8725_e237f29a1e48.slice/crio-47faa97993d94c2cd082692fdf6a62c081a49ac9b0d222a43b47524ceec87212 WatchSource:0}: Error finding container 47faa97993d94c2cd082692fdf6a62c081a49ac9b0d222a43b47524ceec87212: Status 404 returned error can't find the container with id 47faa97993d94c2cd082692fdf6a62c081a49ac9b0d222a43b47524ceec87212
Jul 23 13:30:00.335753 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.335736    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 13:30:00.413219 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:00.413184    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn" event={"ID":"b41c1db3-127a-4fcb-8725-e237f29a1e48","Type":"ContainerStarted","Data":"47faa97993d94c2cd082692fdf6a62c081a49ac9b0d222a43b47524ceec87212"}
Jul 23 13:30:03.433459 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:03.433419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn" event={"ID":"b41c1db3-127a-4fcb-8725-e237f29a1e48","Type":"ContainerStarted","Data":"60c681979ab0d9e1d9a52c7b632ddfde0d0d64605ec6956a4865c8ab96649586"}
Jul 23 13:30:03.451919 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:03.451867    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn" podStartSLOduration=1.095492993 podStartE2EDuration="3.451853333s" totalImagesPullingTime="2.35636034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:30:03.450598626 +0000 UTC m=+733.826549405" watchObservedRunningTime="2026-07-23 13:30:03.451853333 +0000 UTC m=+733.827804110"
Jul 23 13:30:04.439749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:04.439715    2559 generic.go:356] "Generic (PLEG): container finished" podID="b41c1db3-127a-4fcb-8725-e237f29a1e48" containerID="60c681979ab0d9e1d9a52c7b632ddfde0d0d64605ec6956a4865c8ab96649586" exitCode=0
Jul 23 13:30:04.440212 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:04.439781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn" event={"ID":"b41c1db3-127a-4fcb-8725-e237f29a1e48","Type":"ContainerDied","Data":"60c681979ab0d9e1d9a52c7b632ddfde0d0d64605ec6956a4865c8ab96649586"}
Jul 23 13:30:05.579871 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:05.579848    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"
Jul 23 13:30:06.126712 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.126681    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-8krpd"]
Jul 23 13:30:06.126896 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.126873    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd" podUID="e982661e-f60b-4b28-b132-067c346fa4d3" containerName="manager" containerID="cri-o://450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335" gracePeriod=30
Jul 23 13:30:06.379151 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.379131    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:30:06.450141 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.450105    2559 generic.go:356] "Generic (PLEG): container finished" podID="e982661e-f60b-4b28-b132-067c346fa4d3" containerID="450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335" exitCode=0
Jul 23 13:30:06.450141 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.450145    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd"
Jul 23 13:30:06.450376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.450187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd" event={"ID":"e982661e-f60b-4b28-b132-067c346fa4d3","Type":"ContainerDied","Data":"450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335"}
Jul 23 13:30:06.450376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.450240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-8krpd" event={"ID":"e982661e-f60b-4b28-b132-067c346fa4d3","Type":"ContainerDied","Data":"164fb93be4c0c5a13e60f46ee8c81532393f2112bab95c7d1073f89fba7c3373"}
Jul 23 13:30:06.450376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.450268    2559 scope.go:117] "RemoveContainer" containerID="450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335"
Jul 23 13:30:06.451694 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.451626    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn" event={"ID":"b41c1db3-127a-4fcb-8725-e237f29a1e48","Type":"ContainerDied","Data":"47faa97993d94c2cd082692fdf6a62c081a49ac9b0d222a43b47524ceec87212"}
Jul 23 13:30:06.451694 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.451656    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-xq6sn"
Jul 23 13:30:06.451997 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.451661    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47faa97993d94c2cd082692fdf6a62c081a49ac9b0d222a43b47524ceec87212"
Jul 23 13:30:06.459258 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.459114    2559 scope.go:117] "RemoveContainer" containerID="450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335"
Jul 23 13:30:06.460650 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:30:06.459453    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335\": container with ID starting with 450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335 not found: ID does not exist" containerID="450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335"
Jul 23 13:30:06.460650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.459493    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335"} err="failed to get container status \"450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335\": rpc error: code = NotFound desc = could not find container \"450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335\": container with ID starting with 450e0ca011ba412bb05273cc781dd71481704e97dc79f982cee00d4b5f257335 not found: ID does not exist"
Jul 23 13:30:06.476869 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.476838    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdbdj\" (UniqueName: \"kubernetes.io/projected/e982661e-f60b-4b28-b132-067c346fa4d3-kube-api-access-xdbdj\") pod \"e982661e-f60b-4b28-b132-067c346fa4d3\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") "
Jul 23 13:30:06.477008 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.476989    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e982661e-f60b-4b28-b132-067c346fa4d3-webhook-cert\") pod \"e982661e-f60b-4b28-b132-067c346fa4d3\" (UID: \"e982661e-f60b-4b28-b132-067c346fa4d3\") "
Jul 23 13:30:06.479052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.479025    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e982661e-f60b-4b28-b132-067c346fa4d3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e982661e-f60b-4b28-b132-067c346fa4d3" (UID: "e982661e-f60b-4b28-b132-067c346fa4d3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:30:06.479139 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.479025    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e982661e-f60b-4b28-b132-067c346fa4d3-kube-api-access-xdbdj" (OuterVolumeSpecName: "kube-api-access-xdbdj") pod "e982661e-f60b-4b28-b132-067c346fa4d3" (UID: "e982661e-f60b-4b28-b132-067c346fa4d3"). InnerVolumeSpecName "kube-api-access-xdbdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:30:06.578300 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.578275    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e982661e-f60b-4b28-b132-067c346fa4d3-webhook-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:30:06.578383 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.578301    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdbdj\" (UniqueName: \"kubernetes.io/projected/e982661e-f60b-4b28-b132-067c346fa4d3-kube-api-access-xdbdj\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:30:06.776810 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.776776    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-8krpd"]
Jul 23 13:30:06.781215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:06.781186    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-8krpd"]
Jul 23 13:30:08.092323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:30:08.092289    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e982661e-f60b-4b28-b132-067c346fa4d3" path="/var/lib/kubelet/pods/e982661e-f60b-4b28-b132-067c346fa4d3/volumes"
Jul 23 13:31:00.952266 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:00.952230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:31:07.637922 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.637890    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-2w62q"]
Jul 23 13:31:07.638295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.638239    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41c1db3-127a-4fcb-8725-e237f29a1e48" containerName="cleanup"
Jul 23 13:31:07.638295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.638249    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41c1db3-127a-4fcb-8725-e237f29a1e48" containerName="cleanup"
Jul 23 13:31:07.638295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.638268    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e982661e-f60b-4b28-b132-067c346fa4d3" containerName="manager"
Jul 23 13:31:07.638295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.638274    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e982661e-f60b-4b28-b132-067c346fa4d3" containerName="manager"
Jul 23 13:31:07.638421 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.638326    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e982661e-f60b-4b28-b132-067c346fa4d3" containerName="manager"
Jul 23 13:31:07.638421 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.638337    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b41c1db3-127a-4fcb-8725-e237f29a1e48" containerName="cleanup"
Jul 23 13:31:07.641250 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.641232    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.644074 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.644051    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 13:31:07.645165 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.645148    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h7qbw\""
Jul 23 13:31:07.645292 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.645153    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 13:31:07.649745 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.649721    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-2w62q"]
Jul 23 13:31:07.716367 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.716328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lh2d\" (UniqueName: \"kubernetes.io/projected/14e9c702-fb61-4281-a2a3-64472d3e41f3-kube-api-access-5lh2d\") pod \"maas-controller-7fdc9c77b7-2w62q\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") " pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.716576 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.716396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14e9c702-fb61-4281-a2a3-64472d3e41f3-webhook-cert\") pod \"maas-controller-7fdc9c77b7-2w62q\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") " pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.817758 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.817719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14e9c702-fb61-4281-a2a3-64472d3e41f3-webhook-cert\") pod \"maas-controller-7fdc9c77b7-2w62q\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") " pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.817940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.817789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lh2d\" (UniqueName: \"kubernetes.io/projected/14e9c702-fb61-4281-a2a3-64472d3e41f3-kube-api-access-5lh2d\") pod \"maas-controller-7fdc9c77b7-2w62q\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") " pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.820333 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.820307    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14e9c702-fb61-4281-a2a3-64472d3e41f3-webhook-cert\") pod \"maas-controller-7fdc9c77b7-2w62q\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") " pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.826875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.826841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lh2d\" (UniqueName: \"kubernetes.io/projected/14e9c702-fb61-4281-a2a3-64472d3e41f3-kube-api-access-5lh2d\") pod \"maas-controller-7fdc9c77b7-2w62q\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") " pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:07.951972 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:07.951873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:08.088857 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:31:08.088822    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e9c702_fb61_4281_a2a3_64472d3e41f3.slice/crio-17293caf34619805877efa03a587b4f1d270123754cd7b34b1b1b667788f8042 WatchSource:0}: Error finding container 17293caf34619805877efa03a587b4f1d270123754cd7b34b1b1b667788f8042: Status 404 returned error can't find the container with id 17293caf34619805877efa03a587b4f1d270123754cd7b34b1b1b667788f8042
Jul 23 13:31:08.092416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:08.092393    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-2w62q"]
Jul 23 13:31:08.745505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:08.745468    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q" event={"ID":"14e9c702-fb61-4281-a2a3-64472d3e41f3","Type":"ContainerStarted","Data":"17293caf34619805877efa03a587b4f1d270123754cd7b34b1b1b667788f8042"}
Jul 23 13:31:09.751135 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:09.751094    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q" event={"ID":"14e9c702-fb61-4281-a2a3-64472d3e41f3","Type":"ContainerStarted","Data":"c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a"}
Jul 23 13:31:09.751505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:09.751162    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:09.773842 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:09.773798    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q" podStartSLOduration=1.8388504 podStartE2EDuration="2.773784327s" totalImagesPullingTime="934.933927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:31:09.772407957 +0000 UTC m=+800.148358735" watchObservedRunningTime="2026-07-23 13:31:09.773784327 +0000 UTC m=+800.149735105"
Jul 23 13:31:15.286270 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:15.286192    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:31:20.759168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:20.759141    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:21.990161 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:21.990125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:31:22.049979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:22.049944    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:31:25.394145 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.394102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:31:25.541417 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.541381    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-2w62q"]
Jul 23 13:31:25.541689 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.541636    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q" podUID="14e9c702-fb61-4281-a2a3-64472d3e41f3" containerName="manager" containerID="cri-o://c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a" gracePeriod=30
Jul 23 13:31:25.797285 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.797261    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:25.827206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.827164    2559 generic.go:356] "Generic (PLEG): container finished" podID="14e9c702-fb61-4281-a2a3-64472d3e41f3" containerID="c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a" exitCode=0
Jul 23 13:31:25.827384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.827229    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q"
Jul 23 13:31:25.827384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.827229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q" event={"ID":"14e9c702-fb61-4281-a2a3-64472d3e41f3","Type":"ContainerDied","Data":"c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a"}
Jul 23 13:31:25.827384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.827339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-2w62q" event={"ID":"14e9c702-fb61-4281-a2a3-64472d3e41f3","Type":"ContainerDied","Data":"17293caf34619805877efa03a587b4f1d270123754cd7b34b1b1b667788f8042"}
Jul 23 13:31:25.827384 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.827369    2559 scope.go:117] "RemoveContainer" containerID="c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a"
Jul 23 13:31:25.836681 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.836663    2559 scope.go:117] "RemoveContainer" containerID="c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a"
Jul 23 13:31:25.836974 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:31:25.836949    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a\": container with ID starting with c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a not found: ID does not exist" containerID="c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a"
Jul 23 13:31:25.837039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.836986    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a"} err="failed to get container status \"c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a\": rpc error: code = NotFound desc = could not find container \"c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a\": container with ID starting with c80573f0db82776d153ae46d6f64caef43dcbaced40d054355bdf9cd2fc6d44a not found: ID does not exist"
Jul 23 13:31:25.876861 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.876827    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14e9c702-fb61-4281-a2a3-64472d3e41f3-webhook-cert\") pod \"14e9c702-fb61-4281-a2a3-64472d3e41f3\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") "
Jul 23 13:31:25.877052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.876882    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lh2d\" (UniqueName: \"kubernetes.io/projected/14e9c702-fb61-4281-a2a3-64472d3e41f3-kube-api-access-5lh2d\") pod \"14e9c702-fb61-4281-a2a3-64472d3e41f3\" (UID: \"14e9c702-fb61-4281-a2a3-64472d3e41f3\") "
Jul 23 13:31:25.879059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.879029    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e9c702-fb61-4281-a2a3-64472d3e41f3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "14e9c702-fb61-4281-a2a3-64472d3e41f3" (UID: "14e9c702-fb61-4281-a2a3-64472d3e41f3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:31:25.879059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.879035    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e9c702-fb61-4281-a2a3-64472d3e41f3-kube-api-access-5lh2d" (OuterVolumeSpecName: "kube-api-access-5lh2d") pod "14e9c702-fb61-4281-a2a3-64472d3e41f3" (UID: "14e9c702-fb61-4281-a2a3-64472d3e41f3"). InnerVolumeSpecName "kube-api-access-5lh2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:31:25.978080 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.978000    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14e9c702-fb61-4281-a2a3-64472d3e41f3-webhook-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:31:25.978080 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:25.978027    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lh2d\" (UniqueName: \"kubernetes.io/projected/14e9c702-fb61-4281-a2a3-64472d3e41f3-kube-api-access-5lh2d\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:31:26.146654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:26.146623    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-2w62q"]
Jul 23 13:31:26.153723 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:26.153695    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-2w62q"]
Jul 23 13:31:26.958412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:26.958382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:31:27.100656 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.100621    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-l2gkt"]
Jul 23 13:31:27.101017 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.101005    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e9c702-fb61-4281-a2a3-64472d3e41f3" containerName="manager"
Jul 23 13:31:27.101057 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.101018    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e9c702-fb61-4281-a2a3-64472d3e41f3" containerName="manager"
Jul 23 13:31:27.101098 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.101090    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e9c702-fb61-4281-a2a3-64472d3e41f3" containerName="manager"
Jul 23 13:31:27.104038 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.104018    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.106894 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.106874    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 13:31:27.106997 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.106960    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 13:31:27.107062 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.107042    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h7qbw\""
Jul 23 13:31:27.114805 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.114781    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-l2gkt"]
Jul 23 13:31:27.187386 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.187355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ffc36b1-87cd-4cc0-ab23-16dd30749088-webhook-cert\") pod \"maas-controller-7fdc9c77b7-l2gkt\" (UID: \"1ffc36b1-87cd-4cc0-ab23-16dd30749088\") " pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.187575 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.187428    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xcpx\" (UniqueName: \"kubernetes.io/projected/1ffc36b1-87cd-4cc0-ab23-16dd30749088-kube-api-access-2xcpx\") pod \"maas-controller-7fdc9c77b7-l2gkt\" (UID: \"1ffc36b1-87cd-4cc0-ab23-16dd30749088\") " pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.288291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.288204    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ffc36b1-87cd-4cc0-ab23-16dd30749088-webhook-cert\") pod \"maas-controller-7fdc9c77b7-l2gkt\" (UID: \"1ffc36b1-87cd-4cc0-ab23-16dd30749088\") " pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.288291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.288260    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xcpx\" (UniqueName: \"kubernetes.io/projected/1ffc36b1-87cd-4cc0-ab23-16dd30749088-kube-api-access-2xcpx\") pod \"maas-controller-7fdc9c77b7-l2gkt\" (UID: \"1ffc36b1-87cd-4cc0-ab23-16dd30749088\") " pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.290861 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.290835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ffc36b1-87cd-4cc0-ab23-16dd30749088-webhook-cert\") pod \"maas-controller-7fdc9c77b7-l2gkt\" (UID: \"1ffc36b1-87cd-4cc0-ab23-16dd30749088\") " pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.297422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.297394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xcpx\" (UniqueName: \"kubernetes.io/projected/1ffc36b1-87cd-4cc0-ab23-16dd30749088-kube-api-access-2xcpx\") pod \"maas-controller-7fdc9c77b7-l2gkt\" (UID: \"1ffc36b1-87cd-4cc0-ab23-16dd30749088\") " pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.415291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.415259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:27.551262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.551176    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fdc9c77b7-l2gkt"]
Jul 23 13:31:27.554841 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:31:27.554810    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffc36b1_87cd_4cc0_ab23_16dd30749088.slice/crio-2edeb8a0dbfd3aeeda1f8b31aa3a557dc60e5058ca5104ff83564f49434a607d WatchSource:0}: Error finding container 2edeb8a0dbfd3aeeda1f8b31aa3a557dc60e5058ca5104ff83564f49434a607d: Status 404 returned error can't find the container with id 2edeb8a0dbfd3aeeda1f8b31aa3a557dc60e5058ca5104ff83564f49434a607d
Jul 23 13:31:27.837956 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:27.837924    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt" event={"ID":"1ffc36b1-87cd-4cc0-ab23-16dd30749088","Type":"ContainerStarted","Data":"2edeb8a0dbfd3aeeda1f8b31aa3a557dc60e5058ca5104ff83564f49434a607d"}
Jul 23 13:31:28.091749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:28.091681    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e9c702-fb61-4281-a2a3-64472d3e41f3" path="/var/lib/kubelet/pods/14e9c702-fb61-4281-a2a3-64472d3e41f3/volumes"
Jul 23 13:31:28.844429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:28.844379    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt" event={"ID":"1ffc36b1-87cd-4cc0-ab23-16dd30749088","Type":"ContainerStarted","Data":"71d904585136b1d35c01a497180ced4daa965553c5b1d77412ae68e4fadb1cdd"}
Jul 23 13:31:28.844655 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:28.844528    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:31:28.864199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:28.864152    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt" podStartSLOduration=1.3260414790000001 podStartE2EDuration="1.864137204s" totalImagesPullingTime="538.095725ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:31:28.863655366 +0000 UTC m=+819.239606145" watchObservedRunningTime="2026-07-23 13:31:28.864137204 +0000 UTC m=+819.240087982"
Jul 23 13:31:39.857325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:31:39.857290    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fdc9c77b7-l2gkt"
Jul 23 13:32:03.609360 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:03.609322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:03.709611 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:03.709578    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:03.832716 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:03.832660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:09.682642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:09.682598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:09.748758 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:09.748730    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:09.852770 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:09.852739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:10.112887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:10.112855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:15.995455 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:15.995419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:32:16.069425 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:32:16.069391    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:33:10.006647 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:33:10.006615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:37:54.953595 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:37:54.953555    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:37:55.038879 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:37:55.038810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:38:18.722567 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:38:18.722513    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:39:46.126327 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:39:46.126287    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:39:46.204659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:39:46.204627    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:40:03.148390 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.148355    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"]
Jul 23 13:40:03.151877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.151859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.154873 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.154846    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-042ee759-kserve-self-signed-certs\""
Jul 23 13:40:03.156121 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.156092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-home\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.156233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.156128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tls-certs\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.156316 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.156262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5k6c\" (UniqueName: \"kubernetes.io/projected/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-kube-api-access-g5k6c\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.156316 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.156310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tmp-dir\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.156596 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.156577    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-dshm\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.156694 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.156610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-model-cache\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.175573 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.175547    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"]
Jul 23 13:40:03.185359 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.185332    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:40:03.257627 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-home\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.257778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257644    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tls-certs\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.257778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257706    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5k6c\" (UniqueName: \"kubernetes.io/projected/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-kube-api-access-g5k6c\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.257778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tmp-dir\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.257925 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-dshm\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.257925 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257823    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-model-cache\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.258027 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.257936    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-home\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.258127 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.258097    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tmp-dir\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.258293 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.258175    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-model-cache\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.260146 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.260126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-dshm\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.260397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.260381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tls-certs\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.266325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.266305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5k6c\" (UniqueName: \"kubernetes.io/projected/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-kube-api-access-g5k6c\") pod \"e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") " pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.462733 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.461360    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:03.602362 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:03.602338    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"]
Jul 23 13:40:03.604303 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:40:03.604275    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dc2bcdc_71ac_457d_b84e_6f4fd61ee635.slice/crio-9e66cd03a3a33dfd2d102b163f9751296f8098d3e228cb1f541ce39c9fdbcebb WatchSource:0}: Error finding container 9e66cd03a3a33dfd2d102b163f9751296f8098d3e228cb1f541ce39c9fdbcebb: Status 404 returned error can't find the container with id 9e66cd03a3a33dfd2d102b163f9751296f8098d3e228cb1f541ce39c9fdbcebb
Jul 23 13:40:04.281738 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:04.281707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx" event={"ID":"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635","Type":"ContainerStarted","Data":"b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04"}
Jul 23 13:40:04.282094 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:04.281748    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx" event={"ID":"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635","Type":"ContainerStarted","Data":"9e66cd03a3a33dfd2d102b163f9751296f8098d3e228cb1f541ce39c9fdbcebb"}
Jul 23 13:40:04.299768 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:04.299712    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx" podStartSLOduration=1.29969631 podStartE2EDuration="1.29969631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:40:04.299288843 +0000 UTC m=+1334.675239622" watchObservedRunningTime="2026-07-23 13:40:04.29969631 +0000 UTC m=+1334.675647087"
Jul 23 13:40:07.141890 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.141854    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"]
Jul 23 13:40:07.146085 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.146062    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.148897 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.148875    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-042ee759-kserve-self-signed-certs\""
Jul 23 13:40:07.157209 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.157180    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"]
Jul 23 13:40:07.165001 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.164974    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:40:07.195029 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.194998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llh6r\" (UniqueName: \"kubernetes.io/projected/ee6f539a-c5a3-461b-a363-d4c8d022a106-kube-api-access-llh6r\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.195144 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.195045    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-tmp-dir\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.195144 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.195129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-model-cache\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.195238 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.195224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-home\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.195276 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.195248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-dshm\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.195276 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.195270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee6f539a-c5a3-461b-a363-d4c8d022a106-tls-certs\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.296617 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.296579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee6f539a-c5a3-461b-a363-d4c8d022a106-tls-certs\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.296816 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.296638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llh6r\" (UniqueName: \"kubernetes.io/projected/ee6f539a-c5a3-461b-a363-d4c8d022a106-kube-api-access-llh6r\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.296816 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.296660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-tmp-dir\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.296816 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.296687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-model-cache\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.296816 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.296777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-home\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.296816 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.296804    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-dshm\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.297184 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.297158    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-model-cache\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.297253 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.297191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-tmp-dir\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.297253 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.297225    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-home\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.299125 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.299106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-dshm\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.299258 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.299242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee6f539a-c5a3-461b-a363-d4c8d022a106-tls-certs\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.304235 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.304217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llh6r\" (UniqueName: \"kubernetes.io/projected/ee6f539a-c5a3-461b-a363-d4c8d022a106-kube-api-access-llh6r\") pod \"e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") " pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.459496 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.459418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:07.596153 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:07.596110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"]
Jul 23 13:40:07.599306 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:40:07.599276    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee6f539a_c5a3_461b_a363_d4c8d022a106.slice/crio-8c3ffb3476864b64dff11ec6bcca5506ff0a4a23c8753b869d68eb25184944bc WatchSource:0}: Error finding container 8c3ffb3476864b64dff11ec6bcca5506ff0a4a23c8753b869d68eb25184944bc: Status 404 returned error can't find the container with id 8c3ffb3476864b64dff11ec6bcca5506ff0a4a23c8753b869d68eb25184944bc
Jul 23 13:40:08.303373 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:08.303337    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p" event={"ID":"ee6f539a-c5a3-461b-a363-d4c8d022a106","Type":"ContainerStarted","Data":"e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b"}
Jul 23 13:40:08.303790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:08.303383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p" event={"ID":"ee6f539a-c5a3-461b-a363-d4c8d022a106","Type":"ContainerStarted","Data":"8c3ffb3476864b64dff11ec6bcca5506ff0a4a23c8753b869d68eb25184944bc"}
Jul 23 13:40:08.327507 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:08.327465    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p" podStartSLOduration=1.32745156 podStartE2EDuration="1.32745156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:40:08.325217861 +0000 UTC m=+1338.701168652" watchObservedRunningTime="2026-07-23 13:40:08.32745156 +0000 UTC m=+1338.703402338"
Jul 23 13:40:13.461994 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:13.461899    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:13.461994 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:13.461956    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:13.474105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:13.474083    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:14.341603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:14.341574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:17.460065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:17.460028    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:17.460580 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:17.460111    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:17.472444 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:17.472428    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:18.362213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:18.362186    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:35.182682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:35.182646    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"]
Jul 23 13:40:35.227316 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:35.227288    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:40:36.741840 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:36.741805    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"]
Jul 23 13:40:36.764820 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:36.764783    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:40:50.203889 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.203848    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx" podUID="3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" containerName="main" containerID="cri-o://b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04" gracePeriod=45
Jul 23 13:40:50.460237 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.460175    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:50.493476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493442    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tls-certs\") pod \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") "
Jul 23 13:40:50.493657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493546    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5k6c\" (UniqueName: \"kubernetes.io/projected/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-kube-api-access-g5k6c\") pod \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") "
Jul 23 13:40:50.493657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493631    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-dshm\") pod \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") "
Jul 23 13:40:50.493750 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493690    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-model-cache\") pod \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") "
Jul 23 13:40:50.493750 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493719    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-home\") pod \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") "
Jul 23 13:40:50.493848 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493773    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tmp-dir\") pod \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\" (UID: \"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635\") "
Jul 23 13:40:50.494018 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.493989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-model-cache" (OuterVolumeSpecName: "model-cache") pod "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" (UID: "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:50.494285 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.494003    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-home" (OuterVolumeSpecName: "home") pod "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" (UID: "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:50.494503 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.494259    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" (UID: "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:50.495881 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.495860    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" (UID: "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:40:50.496410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.496383    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-dshm" (OuterVolumeSpecName: "dshm") pod "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" (UID: "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:50.496410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.496383    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-kube-api-access-g5k6c" (OuterVolumeSpecName: "kube-api-access-g5k6c") pod "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" (UID: "3dc2bcdc-71ac-457d-b84e-6f4fd61ee635"). InnerVolumeSpecName "kube-api-access-g5k6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:40:50.511223 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.511191    2559 generic.go:356] "Generic (PLEG): container finished" podID="3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" containerID="b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04" exitCode=0
Jul 23 13:40:50.511357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.511243    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"
Jul 23 13:40:50.511357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.511272    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx" event={"ID":"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635","Type":"ContainerDied","Data":"b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04"}
Jul 23 13:40:50.511357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.511325    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx" event={"ID":"3dc2bcdc-71ac-457d-b84e-6f4fd61ee635","Type":"ContainerDied","Data":"9e66cd03a3a33dfd2d102b163f9751296f8098d3e228cb1f541ce39c9fdbcebb"}
Jul 23 13:40:50.511357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.511347    2559 scope.go:117] "RemoveContainer" containerID="b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04"
Jul 23 13:40:50.521677 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.521659    2559 scope.go:117] "RemoveContainer" containerID="b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04"
Jul 23 13:40:50.521980 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:40:50.521958    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04\": container with ID starting with b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04 not found: ID does not exist" containerID="b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04"
Jul 23 13:40:50.522044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.521991    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04"} err="failed to get container status \"b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04\": rpc error: code = NotFound desc = could not find container \"b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04\": container with ID starting with b5fed287fc350ef5db31323118f67272fd4fe2d712f4ea7e2542fc8b26ea5d04 not found: ID does not exist"
Jul 23 13:40:50.536558 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.536509    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"]
Jul 23 13:40:50.540795 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.540773    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-042ee759-kserve-664fd948f5-5t5wx"]
Jul 23 13:40:50.595040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.595001    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:50.595040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.595042    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:50.595232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.595058    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5k6c\" (UniqueName: \"kubernetes.io/projected/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-kube-api-access-g5k6c\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:50.595232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.595068    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:50.595232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.595077    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:50.595232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:50.595084    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:51.763066 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:51.763022    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p" podUID="ee6f539a-c5a3-461b-a363-d4c8d022a106" containerName="main" containerID="cri-o://e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b" gracePeriod=45
Jul 23 13:40:52.030503 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.030475    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:52.092487 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.092453    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" path="/var/lib/kubelet/pods/3dc2bcdc-71ac-457d-b84e-6f4fd61ee635/volumes"
Jul 23 13:40:52.108724 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.108696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-dshm\") pod \"ee6f539a-c5a3-461b-a363-d4c8d022a106\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") "
Jul 23 13:40:52.108849 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.108737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llh6r\" (UniqueName: \"kubernetes.io/projected/ee6f539a-c5a3-461b-a363-d4c8d022a106-kube-api-access-llh6r\") pod \"ee6f539a-c5a3-461b-a363-d4c8d022a106\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") "
Jul 23 13:40:52.108849 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.108777    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee6f539a-c5a3-461b-a363-d4c8d022a106-tls-certs\") pod \"ee6f539a-c5a3-461b-a363-d4c8d022a106\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") "
Jul 23 13:40:52.108849 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.108793    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-tmp-dir\") pod \"ee6f539a-c5a3-461b-a363-d4c8d022a106\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") "
Jul 23 13:40:52.109015 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.108851    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-home\") pod \"ee6f539a-c5a3-461b-a363-d4c8d022a106\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") "
Jul 23 13:40:52.109015 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.108883    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-model-cache\") pod \"ee6f539a-c5a3-461b-a363-d4c8d022a106\" (UID: \"ee6f539a-c5a3-461b-a363-d4c8d022a106\") "
Jul 23 13:40:52.109152 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.109105    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee6f539a-c5a3-461b-a363-d4c8d022a106" (UID: "ee6f539a-c5a3-461b-a363-d4c8d022a106"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:52.109215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.109136    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-home" (OuterVolumeSpecName: "home") pod "ee6f539a-c5a3-461b-a363-d4c8d022a106" (UID: "ee6f539a-c5a3-461b-a363-d4c8d022a106"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:52.109215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.109163    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee6f539a-c5a3-461b-a363-d4c8d022a106" (UID: "ee6f539a-c5a3-461b-a363-d4c8d022a106"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:52.109325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.109236    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:52.109325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.109255    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:52.111039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.111019    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee6f539a-c5a3-461b-a363-d4c8d022a106-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee6f539a-c5a3-461b-a363-d4c8d022a106" (UID: "ee6f539a-c5a3-461b-a363-d4c8d022a106"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:40:52.111117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.111068    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-dshm" (OuterVolumeSpecName: "dshm") pod "ee6f539a-c5a3-461b-a363-d4c8d022a106" (UID: "ee6f539a-c5a3-461b-a363-d4c8d022a106"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:40:52.111153 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.111116    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee6f539a-c5a3-461b-a363-d4c8d022a106-kube-api-access-llh6r" (OuterVolumeSpecName: "kube-api-access-llh6r") pod "ee6f539a-c5a3-461b-a363-d4c8d022a106" (UID: "ee6f539a-c5a3-461b-a363-d4c8d022a106"). InnerVolumeSpecName "kube-api-access-llh6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:40:52.210403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.210371    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee6f539a-c5a3-461b-a363-d4c8d022a106-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:52.210403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.210402    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:52.210403 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.210412    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee6f539a-c5a3-461b-a363-d4c8d022a106-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:52.210673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.210422    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llh6r\" (UniqueName: \"kubernetes.io/projected/ee6f539a-c5a3-461b-a363-d4c8d022a106-kube-api-access-llh6r\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:40:52.522277 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.522247    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee6f539a-c5a3-461b-a363-d4c8d022a106" containerID="e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b" exitCode=0
Jul 23 13:40:52.522463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.522288    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"
Jul 23 13:40:52.522463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.522337    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p" event={"ID":"ee6f539a-c5a3-461b-a363-d4c8d022a106","Type":"ContainerDied","Data":"e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b"}
Jul 23 13:40:52.522463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.522374    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p" event={"ID":"ee6f539a-c5a3-461b-a363-d4c8d022a106","Type":"ContainerDied","Data":"8c3ffb3476864b64dff11ec6bcca5506ff0a4a23c8753b869d68eb25184944bc"}
Jul 23 13:40:52.522463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.522392    2559 scope.go:117] "RemoveContainer" containerID="e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b"
Jul 23 13:40:52.531999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.531983    2559 scope.go:117] "RemoveContainer" containerID="e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b"
Jul 23 13:40:52.532256 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:40:52.532240    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b\": container with ID starting with e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b not found: ID does not exist" containerID="e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b"
Jul 23 13:40:52.532317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.532263    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b"} err="failed to get container status \"e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b\": rpc error: code = NotFound desc = could not find container \"e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b\": container with ID starting with e5d945084d2b4a49b3dd1d85339720aae2ca1f13b69cabe497121bef7c2f298b not found: ID does not exist"
Jul 23 13:40:52.544260 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.544230    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"]
Jul 23 13:40:52.546441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:52.546407    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-042ee759-kserve-568cd4b59f-wjz8p"]
Jul 23 13:40:54.097103 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:40:54.097004    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee6f539a-c5a3-461b-a363-d4c8d022a106" path="/var/lib/kubelet/pods/ee6f539a-c5a3-461b-a363-d4c8d022a106/volumes"
Jul 23 13:41:58.291619 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:41:58.291471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:43:51.421756 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:43:51.421723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:12.269760 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:12.269725    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:15.619817 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:15.619780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:15.721252 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:15.721222    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:16.651855 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:16.651819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:22.110603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.110569    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:22.184118 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.184082    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:22.412269 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.412181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:22.691063 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.690972    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"]
Jul 23 13:44:22.691459 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.691441    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" containerName="main"
Jul 23 13:44:22.691565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.691462    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" containerName="main"
Jul 23 13:44:22.691565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.691479    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee6f539a-c5a3-461b-a363-d4c8d022a106" containerName="main"
Jul 23 13:44:22.691565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.691487    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6f539a-c5a3-461b-a363-d4c8d022a106" containerName="main"
Jul 23 13:44:22.691730 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.691629    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee6f539a-c5a3-461b-a363-d4c8d022a106" containerName="main"
Jul 23 13:44:22.691730 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.691650    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dc2bcdc-71ac-457d-b84e-6f4fd61ee635" containerName="main"
Jul 23 13:44:22.694780 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.694759    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.698187 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.698170    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7d6a9a51-openshift-default-dockercfg-4n6g5\""
Jul 23 13:44:22.718682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.718644    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"]
Jul 23 13:44:22.719805 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.719779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:22.762557 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-socket\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762557 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-certs\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762612    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30bd251f-d2da-48fb-83a8-c50a1aacf622-istiod-ca-cert\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-token\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-data\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-credential-socket\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-envoy\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77spm\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-kube-api-access-77spm\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.762991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.762815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-podinfo\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864159 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-credential-socket\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-envoy\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77spm\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-kube-api-access-77spm\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-podinfo\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864299    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-socket\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-certs\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864672 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30bd251f-d2da-48fb-83a8-c50a1aacf622-istiod-ca-cert\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864672 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-token\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864672 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864430    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-data\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864672 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864524    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-credential-socket\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864890 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864792    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-data\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864890 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864826    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-socket\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.864890 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.864859    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-certs\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.865218 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.865150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30bd251f-d2da-48fb-83a8-c50a1aacf622-istiod-ca-cert\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.867052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.867026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-envoy\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.867195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.867084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-podinfo\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.873155 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.873125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77spm\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-kube-api-access-77spm\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.874330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.874307    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-token\") pod \"e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") " pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:22.925376 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:22.925341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:23.005036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.004946    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:23.150716 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.150687    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"]
Jul 23 13:44:23.152898 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:44:23.152869    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30bd251f_d2da_48fb_83a8_c50a1aacf622.slice/crio-8e3bc2b49404da94e2b8d7b8ab7be05f024bd9edbae24bc0a67e243be6986295 WatchSource:0}: Error finding container 8e3bc2b49404da94e2b8d7b8ab7be05f024bd9edbae24bc0a67e243be6986295: Status 404 returned error can't find the container with id 8e3bc2b49404da94e2b8d7b8ab7be05f024bd9edbae24bc0a67e243be6986295
Jul 23 13:44:23.154946 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.154919    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:44:23.155065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.154981    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:44:23.155065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.155028    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:44:23.519778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.519741    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" event={"ID":"30bd251f-d2da-48fb-83a8-c50a1aacf622","Type":"ContainerStarted","Data":"8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"}
Jul 23 13:44:23.519975 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.519803    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" event={"ID":"30bd251f-d2da-48fb-83a8-c50a1aacf622","Type":"ContainerStarted","Data":"8e3bc2b49404da94e2b8d7b8ab7be05f024bd9edbae24bc0a67e243be6986295"}
Jul 23 13:44:23.544797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:23.544743    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podStartSLOduration=1.54472757 podStartE2EDuration="1.54472757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:44:23.542743498 +0000 UTC m=+1593.918694277" watchObservedRunningTime="2026-07-23 13:44:23.54472757 +0000 UTC m=+1593.920678347"
Jul 23 13:44:24.005483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:24.005449    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:24.010720 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:24.010690    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:24.524985 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:24.524959    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:24.526019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:24.525994    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:24.926447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:24.926401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:25.037215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:25.037183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:25.129006 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:25.128869    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:25.765832 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:25.765798    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:25.865723 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:25.865689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:40.566687 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:40.566649    2559 patch_prober.go:28] interesting pod/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 13:44:40.567089 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:40.566737    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 13:44:41.610973 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:41.610938    2559 generic.go:356] "Generic (PLEG): container finished" podID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerID="8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76" exitCode=0
Jul 23 13:44:41.611460 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:41.610981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" event={"ID":"30bd251f-d2da-48fb-83a8-c50a1aacf622","Type":"ContainerDied","Data":"8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"}
Jul 23 13:44:41.611460 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:41.611396    2559 scope.go:117] "RemoveContainer" containerID="8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"
Jul 23 13:44:41.612452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:41.612424    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:44:41.612514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:41.612493    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:44:41.612600 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:41.612528    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:44:42.618423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:42.618391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" event={"ID":"30bd251f-d2da-48fb-83a8-c50a1aacf622","Type":"ContainerStarted","Data":"326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a"}
Jul 23 13:44:43.005972 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:43.005881    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:43.006140 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:43.006119    2559 patch_prober.go:28] interesting pod/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 23 13:44:43.006189 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:43.006172    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 23 13:44:44.006128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:44.006088    2559 patch_prober.go:28] interesting pod/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 23 13:44:44.006629 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:44.006154    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 23 13:44:45.006366 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:45.006323    2559 patch_prober.go:28] interesting pod/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 23 13:44:45.006794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:45.006384    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 23 13:44:46.005944 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:46.005900    2559 patch_prober.go:28] interesting pod/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 23 13:44:46.006143 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:46.005961    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 23 13:44:46.565877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:46.565838    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:44:47.009234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:47.009199    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:47.009593 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:47.009564    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:44:47.010388 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:44:47.010371    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:45:01.265911 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:01.265872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:05.738663 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:05.738624    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:05.877432 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:05.877375    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"]
Jul 23 13:45:05.877963 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:05.877912    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy" containerID="cri-o://326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a" gracePeriod=30
Jul 23 13:45:05.906422 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:05.906392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:06.179362 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.179327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:06.443987 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.443872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:06.537226 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.537183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:06.653824 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.653790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:06.960161 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.960122    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"]
Jul 23 13:45:06.963886 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.963863    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:06.966967 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.966947    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-38128cd5-gw-openshift-default-dockercfg-bxkkx\""
Jul 23 13:45:06.980952 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:06.980920    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"]
Jul 23 13:45:07.026309 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.026270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:07.068952 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.068909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj9m5\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-kube-api-access-wj9m5\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.068952 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.068953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-credential-socket\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069007    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-data\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-token\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069072    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-socket\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-envoy\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069105    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-podinfo\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069150    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istiod-ca-cert\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.069387 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.069176    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-certs\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.145767 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.145732    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:07.170421 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istiod-ca-cert\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-certs\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj9m5\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-kube-api-access-wj9m5\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-credential-socket\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-data\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-token\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170950 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170683    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-socket\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170950 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-envoy\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.170950 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.170741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-podinfo\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.171109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.171027    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-credential-socket\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.171109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.171055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-certs\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.171109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.171095    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-socket\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.171313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.171285    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istiod-ca-cert\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.171388 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.171349    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-data\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.173225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.173203    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-envoy\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.173774 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.173751    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-podinfo\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.180071 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.180045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj9m5\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-kube-api-access-wj9m5\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.180817 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.180795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-token\") pod \"e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") " pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.274849 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.274749    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:07.426875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.426849    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"]
Jul 23 13:45:07.428592 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:45:07.428562    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fb48b1_4e50_4bd2_a4ac_cf5b82c47a83.slice/crio-994b4e1472d97355c3d925495a8c6f9ba1d756de7f7dc667174668ab3a8a9b6b WatchSource:0}: Error finding container 994b4e1472d97355c3d925495a8c6f9ba1d756de7f7dc667174668ab3a8a9b6b: Status 404 returned error can't find the container with id 994b4e1472d97355c3d925495a8c6f9ba1d756de7f7dc667174668ab3a8a9b6b
Jul 23 13:45:07.430879 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.430847    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:07.430978 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.430911    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:07.430978 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.430940    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:07.746887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.746851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" event={"ID":"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83","Type":"ContainerStarted","Data":"db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"}
Jul 23 13:45:07.747059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.746904    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" event={"ID":"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83","Type":"ContainerStarted","Data":"994b4e1472d97355c3d925495a8c6f9ba1d756de7f7dc667174668ab3a8a9b6b"}
Jul 23 13:45:07.771400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:07.771350    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podStartSLOduration=1.771334227 podStartE2EDuration="1.771334227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:45:07.769837039 +0000 UTC m=+1638.145787829" watchObservedRunningTime="2026-07-23 13:45:07.771334227 +0000 UTC m=+1638.147285004"
Jul 23 13:45:08.275611 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:08.275571    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:08.280489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:08.280461    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:08.752414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:08.752381    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:08.753417 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:08.753396    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:09.264870 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:09.264835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:09.374260 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:09.374230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:09.874671 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:09.874608    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:09.969484 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:09.969453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:10.068631 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:10.068583    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:11.138673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.138606    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:45:11.218841 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.218813    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-token\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.218861    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-podinfo\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.218903    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77spm\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-kube-api-access-77spm\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.218921    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30bd251f-d2da-48fb-83a8-c50a1aacf622-istiod-ca-cert\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.218958    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-data\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.218983    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-envoy\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219303 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219033    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-socket\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219303 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219067    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-certs\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219303 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219087    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-credential-socket\") pod \"30bd251f-d2da-48fb-83a8-c50a1aacf622\" (UID: \"30bd251f-d2da-48fb-83a8-c50a1aacf622\") "
Jul 23 13:45:11.219455 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219346    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30bd251f-d2da-48fb-83a8-c50a1aacf622-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:45:11.219583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219527    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:11.219705 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219583    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:11.219705 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219667    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:11.219953 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.219829    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-data" (OuterVolumeSpecName: "istio-data") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:11.221325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.221299    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-token" (OuterVolumeSpecName: "istio-token") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:45:11.221525 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.221493    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:11.221785 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.221758    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:45:11.221900 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.221785    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-kube-api-access-77spm" (OuterVolumeSpecName: "kube-api-access-77spm") pod "30bd251f-d2da-48fb-83a8-c50a1aacf622" (UID: "30bd251f-d2da-48fb-83a8-c50a1aacf622"). InnerVolumeSpecName "kube-api-access-77spm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:45:11.320873 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320827    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.320873 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320862    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.320873 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320877    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.321124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320889    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.321124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320901    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.321124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320912    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77spm\" (UniqueName: \"kubernetes.io/projected/30bd251f-d2da-48fb-83a8-c50a1aacf622-kube-api-access-77spm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.321124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320924    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30bd251f-d2da-48fb-83a8-c50a1aacf622-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.321124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320935    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.321124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.320946    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30bd251f-d2da-48fb-83a8-c50a1aacf622-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:11.768142 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.768101    2559 generic.go:356] "Generic (PLEG): container finished" podID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerID="326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a" exitCode=0
Jul 23 13:45:11.768331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.768156    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" event={"ID":"30bd251f-d2da-48fb-83a8-c50a1aacf622","Type":"ContainerDied","Data":"326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a"}
Jul 23 13:45:11.768331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.768171    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"
Jul 23 13:45:11.768331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.768202    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r" event={"ID":"30bd251f-d2da-48fb-83a8-c50a1aacf622","Type":"ContainerDied","Data":"8e3bc2b49404da94e2b8d7b8ab7be05f024bd9edbae24bc0a67e243be6986295"}
Jul 23 13:45:11.768331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.768224    2559 scope.go:117] "RemoveContainer" containerID="326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a"
Jul 23 13:45:11.777971 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.777949    2559 scope.go:117] "RemoveContainer" containerID="8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"
Jul 23 13:45:11.787823 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.787785    2559 scope.go:117] "RemoveContainer" containerID="326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a"
Jul 23 13:45:11.788231 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:45:11.788200    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a\": container with ID starting with 326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a not found: ID does not exist" containerID="326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a"
Jul 23 13:45:11.788231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.788242    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a"} err="failed to get container status \"326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a\": rpc error: code = NotFound desc = could not find container \"326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a\": container with ID starting with 326bc8b2b6cce3b8ccd312d9e7c6e3c39247a563d15e04569ec4ee06acaddc3a not found: ID does not exist"
Jul 23 13:45:11.788231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.788268    2559 scope.go:117] "RemoveContainer" containerID="8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"
Jul 23 13:45:11.788655 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:45:11.788634    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76\": container with ID starting with 8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76 not found: ID does not exist" containerID="8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"
Jul 23 13:45:11.788720 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.788667    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76"} err="failed to get container status \"8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76\": rpc error: code = NotFound desc = could not find container \"8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76\": container with ID starting with 8ba17bfee32d116f459ec3674133e4ccf5dd1a543cc1ec5a0bea58a1caaaef76 not found: ID does not exist"
Jul 23 13:45:11.790862 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.790842    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"]
Jul 23 13:45:11.797429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:11.797407    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7d6a9a51-openshift-default-56b56694c9-ss75r"]
Jul 23 13:45:12.092087 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:12.092056    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" path="/var/lib/kubelet/pods/30bd251f-d2da-48fb-83a8-c50a1aacf622/volumes"
Jul 23 13:45:24.073790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.073663    2559 patch_prober.go:28] interesting pod/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:39414->10.133.0.49:15021: read: connection reset by peer" start-of-body=
Jul 23 13:45:24.073790 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.073724    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:39414->10.133.0.49:15021: read: connection reset by peer"
Jul 23 13:45:24.837155 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.837120    2559 generic.go:356] "Generic (PLEG): container finished" podID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerID="db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588" exitCode=0
Jul 23 13:45:24.837360 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.837199    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" event={"ID":"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83","Type":"ContainerDied","Data":"db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"}
Jul 23 13:45:24.837660 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.837642    2559 scope.go:117] "RemoveContainer" containerID="db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"
Jul 23 13:45:24.838735 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.838708    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:24.838820 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.838802    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:24.838862 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:24.838849    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:25.844214 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:25.844181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" event={"ID":"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83","Type":"ContainerStarted","Data":"fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3"}
Jul 23 13:45:26.275684 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:26.275601    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:26.275907 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:26.275886    2559 patch_prober.go:28] interesting pod/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 13:45:26.275961 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:26.275934    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 13:45:27.275354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:27.275317    2559 patch_prober.go:28] interesting pod/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 13:45:27.275775 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:27.275370    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 13:45:28.275776 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:28.275733    2559 patch_prober.go:28] interesting pod/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 13:45:28.276239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:28.275799    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 13:45:29.275372 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:29.275329    2559 patch_prober.go:28] interesting pod/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 13:45:29.275630 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:29.275398    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 13:45:30.278592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:30.278553    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:30.278953 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:30.278854    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:30.279444 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:30.279428    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:44.957833 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:44.957800    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:49.405401 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:49.405367    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:49.547977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:49.547937    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"]
Jul 23 13:45:49.548224 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:49.548173    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy" containerID="cri-o://fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3" gracePeriod=30
Jul 23 13:45:49.572369 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:49.572341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:49.836775 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:49.836741    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:50.075052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.075013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:50.486162 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.486129    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"]
Jul 23 13:45:50.486661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.486489    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy"
Jul 23 13:45:50.486661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.486500    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy"
Jul 23 13:45:50.486661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.486609    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy"
Jul 23 13:45:50.486661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.486621    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy"
Jul 23 13:45:50.489574 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.489555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.493182 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.493164    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-698a9d15-openshift-default-dockercfg-cnzft\""
Jul 23 13:45:50.509320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.509293    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"]
Jul 23 13:45:50.557469 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.557439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:50.589120 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7579573-2231-4faf-842e-5c97bbec47e3-istiod-ca-cert\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589326 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-envoy\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589326 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589202    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-data\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589326 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-socket\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589326 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589257    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-certs\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589326 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-255jl\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-kube-api-access-255jl\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589709 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-credential-socket\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589709 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7579573-2231-4faf-842e-5c97bbec47e3-istio-podinfo\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.589709 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.589500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-istio-token\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690692    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7579573-2231-4faf-842e-5c97bbec47e3-istio-podinfo\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-istio-token\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690750    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7579573-2231-4faf-842e-5c97bbec47e3-istiod-ca-cert\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-envoy\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-data\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690838    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-socket\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690855    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-certs\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-255jl\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-kube-api-access-255jl\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.690980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.690937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-credential-socket\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.691458 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.691431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-data\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.691685 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.691634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-certs\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.691685 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.691659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-credential-socket\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.692030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.691691    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-socket\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.692105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.692078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7579573-2231-4faf-842e-5c97bbec47e3-istiod-ca-cert\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.693491 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.693471    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-envoy\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.693721 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.693703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7579573-2231-4faf-842e-5c97bbec47e3-istio-podinfo\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.699733 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.699705    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-istio-token\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.699813 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.699707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-255jl\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-kube-api-access-255jl\") pod \"e2e-ait-698a9d15-openshift-default-c8d654d45-x826j\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") " pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.713702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.713679    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:50.799320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.799240    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:50.952893 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.952867    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"]
Jul 23 13:45:50.954756 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:45:50.954723    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7579573_2231_4faf_842e_5c97bbec47e3.slice/crio-dadce2c6825dbf4a27dfe32c065499e615dd61396b63e7594a1c185cf999885d WatchSource:0}: Error finding container dadce2c6825dbf4a27dfe32c065499e615dd61396b63e7594a1c185cf999885d: Status 404 returned error can't find the container with id dadce2c6825dbf4a27dfe32c065499e615dd61396b63e7594a1c185cf999885d
Jul 23 13:45:50.956806 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.956775    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:50.956894 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.956838    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:50.956894 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.956867    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:45:50.969176 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:50.969154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" event={"ID":"e7579573-2231-4faf-842e-5c97bbec47e3","Type":"ContainerStarted","Data":"dadce2c6825dbf4a27dfe32c065499e615dd61396b63e7594a1c185cf999885d"}
Jul 23 13:45:51.976254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:51.976213    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" event={"ID":"e7579573-2231-4faf-842e-5c97bbec47e3","Type":"ContainerStarted","Data":"495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"}
Jul 23 13:45:52.001218 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:52.001162    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podStartSLOduration=2.001143222 podStartE2EDuration="2.001143222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:45:51.999913195 +0000 UTC m=+1682.375863974" watchObservedRunningTime="2026-07-23 13:45:52.001143222 +0000 UTC m=+1682.377093999"
Jul 23 13:45:52.800232 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:52.800194    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:52.805202 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:52.805180    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:52.981459 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:52.981431    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:52.982419 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:52.982399    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:45:53.488334 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:53.488300    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:53.604265 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:53.604229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:53.710346 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:53.710277    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:54.149550 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.149506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:54.277803 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.277760    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:54.391808 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.391772    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:45:54.816888 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.816832    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:54.931786 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931752    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj9m5\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-kube-api-access-wj9m5\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.931977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931806    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-credential-socket\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.931977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931827    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-socket\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.931977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931849    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-podinfo\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.931977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931891    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-envoy\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.931977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931920    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-certs\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.931977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931964    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-data\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.932313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.931987    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-token\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.932313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932074    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istiod-ca-cert\") pod \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\" (UID: \"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83\") "
Jul 23 13:45:54.932313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932117    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:54.932313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932132    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:54.932313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932226    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:54.932592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932407    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:54.932592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932429    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:54.932592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932468    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:54.932592 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.932502    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:45:54.933177 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.933071    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-data" (OuterVolumeSpecName: "istio-data") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:54.934277 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.934229    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-kube-api-access-wj9m5" (OuterVolumeSpecName: "kube-api-access-wj9m5") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "kube-api-access-wj9m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:45:54.934402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.934279    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:45:54.934402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.934334    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-token" (OuterVolumeSpecName: "istio-token") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:45:54.934653 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.934625    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" (UID: "03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:45:54.991874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.991839    2559 generic.go:356] "Generic (PLEG): container finished" podID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerID="fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3" exitCode=0
Jul 23 13:45:54.992022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.991888    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"
Jul 23 13:45:54.992022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.991937    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" event={"ID":"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83","Type":"ContainerDied","Data":"fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3"}
Jul 23 13:45:54.992022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.991991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg" event={"ID":"03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83","Type":"ContainerDied","Data":"994b4e1472d97355c3d925495a8c6f9ba1d756de7f7dc667174668ab3a8a9b6b"}
Jul 23 13:45:54.992022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:54.992013    2559 scope.go:117] "RemoveContainer" containerID="fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3"
Jul 23 13:45:55.001505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.001486    2559 scope.go:117] "RemoveContainer" containerID="db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"
Jul 23 13:45:55.009509 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.009493    2559 scope.go:117] "RemoveContainer" containerID="fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3"
Jul 23 13:45:55.009793 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:45:55.009774    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3\": container with ID starting with fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3 not found: ID does not exist" containerID="fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3"
Jul 23 13:45:55.009843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.009802    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3"} err="failed to get container status \"fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3\": rpc error: code = NotFound desc = could not find container \"fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3\": container with ID starting with fa0c19a7ba188f1767ea8a00c61f9b6fb0885e0487809a2da08bdefe6f6b83b3 not found: ID does not exist"
Jul 23 13:45:55.009843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.009819    2559 scope.go:117] "RemoveContainer" containerID="db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"
Jul 23 13:45:55.010067 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:45:55.010048    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588\": container with ID starting with db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588 not found: ID does not exist" containerID="db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"
Jul 23 13:45:55.010108 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.010074    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588"} err="failed to get container status \"db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588\": rpc error: code = NotFound desc = could not find container \"db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588\": container with ID starting with db708167e84c6aa5751764f3f725d298a792b891d1d19ae82c6e574260b77588 not found: ID does not exist"
Jul 23 13:45:55.018364 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.018337    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"]
Jul 23 13:45:55.026152 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.026129    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-38128cd5-gw-openshift-default-5f7d76f57f-kmnjg"]
Jul 23 13:45:55.033447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.033421    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:55.033447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.033445    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj9m5\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-kube-api-access-wj9m5\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:55.033589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.033456    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:55.033589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.033466    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:55.033589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.033474    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:55.033589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:55.033481    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:45:56.091972 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:45:56.091935    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" path="/var/lib/kubelet/pods/03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83/volumes"
Jul 23 13:46:00.516451 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:00.516417    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:07.983301 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:07.983264    2559 patch_prober.go:28] interesting pod/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 13:46:07.983798 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:07.983324    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 13:46:08.062652 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:08.062615    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7579573-2231-4faf-842e-5c97bbec47e3" containerID="495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf" exitCode=0
Jul 23 13:46:08.062652 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:08.062654    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" event={"ID":"e7579573-2231-4faf-842e-5c97bbec47e3","Type":"ContainerDied","Data":"495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"}
Jul 23 13:46:08.063065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:08.063049    2559 scope.go:117] "RemoveContainer" containerID="495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"
Jul 23 13:46:08.064040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:08.064014    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:08.064099 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:08.064086    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:08.064136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:08.064122    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:09.070320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:09.070289    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" event={"ID":"e7579573-2231-4faf-842e-5c97bbec47e3","Type":"ContainerStarted","Data":"b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363"}
Jul 23 13:46:09.800648 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:09.800616    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:46:09.801056 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:09.800934    2559 patch_prober.go:28] interesting pod/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 13:46:09.801285 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:09.801258    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 13:46:10.800992 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:10.800944    2559 patch_prober.go:28] interesting pod/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 13:46:10.801458 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:10.801019    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 13:46:11.800474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:11.800371    2559 patch_prober.go:28] interesting pod/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 13:46:11.800474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:11.800441    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 13:46:12.800177 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:12.800128    2559 patch_prober.go:28] interesting pod/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 13:46:12.800805 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:12.800771    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 13:46:13.803856 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:13.803822    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:46:13.804240 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:13.804106    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:46:13.804728 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:13.804708    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:46:29.892245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:29.892208    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:34.462949 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:34.462898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:35.788411 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:35.788374    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"]
Jul 23 13:46:35.788852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:35.788634    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy" containerID="cri-o://b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363" gracePeriod=30
Jul 23 13:46:35.810492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:35.810464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:36.061927 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.061833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:36.310658 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.310628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:36.641249 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641219    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"]
Jul 23 13:46:36.641643 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641629    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy"
Jul 23 13:46:36.641702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641645    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy"
Jul 23 13:46:36.641702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641655    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy"
Jul 23 13:46:36.641702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641661    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy"
Jul 23 13:46:36.641702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641678    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy"
Jul 23 13:46:36.641702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641683    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30bd251f-d2da-48fb-83a8-c50a1aacf622" containerName="istio-proxy"
Jul 23 13:46:36.641881 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.641741    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy"
Jul 23 13:46:36.644786 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.644770    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.649103 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.649081    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-dceaf465-openshift-default-dockercfg-qj8lg\""
Jul 23 13:46:36.661580 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.661548    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"]
Jul 23 13:46:36.665301 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.665279    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:36.750099 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-certs\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-socket\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-credential-socket\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f074884-4723-4515-8a03-ee79d1debbc5-istiod-ca-cert\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750302    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-data\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750504 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750325    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9txbm\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-kube-api-access-9txbm\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750504 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-istio-token\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750504 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-envoy\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.750504 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.750435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f074884-4723-4515-8a03-ee79d1debbc5-istio-podinfo\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.850982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.850947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-socket\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.850982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.850992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-credential-socket\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f074884-4723-4515-8a03-ee79d1debbc5-istiod-ca-cert\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-data\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9txbm\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-kube-api-access-9txbm\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-istio-token\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-envoy\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f074884-4723-4515-8a03-ee79d1debbc5-istio-podinfo\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-certs\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851413    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-socket\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-credential-socket\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851868 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851567    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-data\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851868 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-certs\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.851868 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.851841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f074884-4723-4515-8a03-ee79d1debbc5-istiod-ca-cert\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.853670 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.853648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-envoy\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.853978 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.853961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f074884-4723-4515-8a03-ee79d1debbc5-istio-podinfo\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.860078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.860051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-istio-token\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.860213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.860190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9txbm\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-kube-api-access-9txbm\") pod \"e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") " pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:36.872382 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.872357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:36.955486 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:36.955390    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:37.122383 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.122360    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"]
Jul 23 13:46:37.124411 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:46:37.124384    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f074884_4723_4515_8a03_ee79d1debbc5.slice/crio-ed7169b34021d6d4a3612eeb8d0108f644a5bc0db0018cdc91d412375d4f1f11 WatchSource:0}: Error finding container ed7169b34021d6d4a3612eeb8d0108f644a5bc0db0018cdc91d412375d4f1f11: Status 404 returned error can't find the container with id ed7169b34021d6d4a3612eeb8d0108f644a5bc0db0018cdc91d412375d4f1f11
Jul 23 13:46:37.126628 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.126599    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:37.126690 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.126662    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:37.126731 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.126701    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:37.206420 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.206340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" event={"ID":"6f074884-4723-4515-8a03-ee79d1debbc5","Type":"ContainerStarted","Data":"29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"}
Jul 23 13:46:37.206420 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.206397    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" event={"ID":"6f074884-4723-4515-8a03-ee79d1debbc5","Type":"ContainerStarted","Data":"ed7169b34021d6d4a3612eeb8d0108f644a5bc0db0018cdc91d412375d4f1f11"}
Jul 23 13:46:37.232317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.232262    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" podStartSLOduration=1.232243926 podStartE2EDuration="1.232243926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:37.230260747 +0000 UTC m=+1727.606211526" watchObservedRunningTime="2026-07-23 13:46:37.232243926 +0000 UTC m=+1727.608194705"
Jul 23 13:46:37.956094 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:37.956059    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:38.621675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:38.621642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:38.723828 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:38.723795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:38.870211 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:38.870167    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:38.960105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:38.960018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:39.176826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:39.176788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:39.223588 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:39.223462    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:39.224400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:39.224382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:39.461137 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:39.461102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:39.592900 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:39.592863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:41.054397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.054334    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:46:41.193489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193440    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-socket\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193493    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-istio-token\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193588    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-credential-socket\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193614    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-data\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193664    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7579573-2231-4faf-842e-5c97bbec47e3-istio-podinfo\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193699    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7579573-2231-4faf-842e-5c97bbec47e3-istiod-ca-cert\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193722    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-certs\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.193783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193756    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-255jl\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-kube-api-access-255jl\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.194088 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193777    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:46:41.194088 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193793    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-envoy\") pod \"e7579573-2231-4faf-842e-5c97bbec47e3\" (UID: \"e7579573-2231-4faf-842e-5c97bbec47e3\") "
Jul 23 13:46:41.194088 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.193904    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:46:41.194255 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.194107    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.194255 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.194125    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.194389 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.194368    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7579573-2231-4faf-842e-5c97bbec47e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:46:41.195018 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.194481    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:46:41.195018 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.194564    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:46:41.196021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.195993    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:46:41.196167 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.196138    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7579573-2231-4faf-842e-5c97bbec47e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:46:41.196247 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.196230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:46:41.196375 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.196359    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-kube-api-access-255jl" (OuterVolumeSpecName: "kube-api-access-255jl") pod "e7579573-2231-4faf-842e-5c97bbec47e3" (UID: "e7579573-2231-4faf-842e-5c97bbec47e3"). InnerVolumeSpecName "kube-api-access-255jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:46:41.235247 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.235217    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7579573-2231-4faf-842e-5c97bbec47e3" containerID="b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363" exitCode=0
Jul 23 13:46:41.235398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.235270    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"
Jul 23 13:46:41.235398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.235302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" event={"ID":"e7579573-2231-4faf-842e-5c97bbec47e3","Type":"ContainerDied","Data":"b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363"}
Jul 23 13:46:41.235398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.235364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j" event={"ID":"e7579573-2231-4faf-842e-5c97bbec47e3","Type":"ContainerDied","Data":"dadce2c6825dbf4a27dfe32c065499e615dd61396b63e7594a1c185cf999885d"}
Jul 23 13:46:41.235398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.235396    2559 scope.go:117] "RemoveContainer" containerID="b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363"
Jul 23 13:46:41.245324 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.245306    2559 scope.go:117] "RemoveContainer" containerID="495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"
Jul 23 13:46:41.253622 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.253604    2559 scope.go:117] "RemoveContainer" containerID="b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363"
Jul 23 13:46:41.253888 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:46:41.253864    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363\": container with ID starting with b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363 not found: ID does not exist" containerID="b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363"
Jul 23 13:46:41.253979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.253893    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363"} err="failed to get container status \"b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363\": rpc error: code = NotFound desc = could not find container \"b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363\": container with ID starting with b5e017b3e1b0aedcb6ffd16914d347e630a4b9e1fa2ec8d9a006a9967b5cc363 not found: ID does not exist"
Jul 23 13:46:41.253979 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.253921    2559 scope.go:117] "RemoveContainer" containerID="495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"
Jul 23 13:46:41.254167 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:46:41.254150    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf\": container with ID starting with 495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf not found: ID does not exist" containerID="495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"
Jul 23 13:46:41.254210 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.254172    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf"} err="failed to get container status \"495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf\": rpc error: code = NotFound desc = could not find container \"495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf\": container with ID starting with 495d15bb0606ac04dc56dcbae99c47fdb3bf2f6e7a1b0d05bd5493ff8547ffdf not found: ID does not exist"
Jul 23 13:46:41.258702 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.258672    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"]
Jul 23 13:46:41.260825 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.260806    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-698a9d15-openshift-default-c8d654d45-x826j"]
Jul 23 13:46:41.295156 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295123    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7579573-2231-4faf-842e-5c97bbec47e3-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.295156 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295154    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7579573-2231-4faf-842e-5c97bbec47e3-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.295352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295170    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.295352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295183    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-255jl\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-kube-api-access-255jl\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.295352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295199    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.295352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295214    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7579573-2231-4faf-842e-5c97bbec47e3-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:41.295352 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:41.295227    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7579573-2231-4faf-842e-5c97bbec47e3-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:46:42.092307 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:42.092272    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" path="/var/lib/kubelet/pods/e7579573-2231-4faf-842e-5c97bbec47e3/volumes"
Jul 23 13:46:49.708009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:49.707970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:49.808255 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:49.808217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:46:53.303173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:53.303083    2559 generic.go:356] "Generic (PLEG): container finished" podID="6f074884-4723-4515-8a03-ee79d1debbc5" containerID="29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7" exitCode=0
Jul 23 13:46:53.303173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:53.303138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" event={"ID":"6f074884-4723-4515-8a03-ee79d1debbc5","Type":"ContainerDied","Data":"29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"}
Jul 23 13:46:53.303699 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:53.303509    2559 scope.go:117] "RemoveContainer" containerID="29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"
Jul 23 13:46:53.304567 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:53.304508    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:53.304663 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:53.304614    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:53.304663 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:53.304648    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:46:54.223920 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:54.223880    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:54.311052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:54.311008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" event={"ID":"6f074884-4723-4515-8a03-ee79d1debbc5","Type":"ContainerStarted","Data":"55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e"}
Jul 23 13:46:54.955773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:54.955740    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:54.956991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:54.956967    2559 patch_prober.go:28] interesting pod/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 23 13:46:54.957109 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:54.957015    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 23 13:46:55.955811 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:55.955778    2559 patch_prober.go:28] interesting pod/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 23 13:46:55.956222 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:55.955832    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 23 13:46:56.956726 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:56.956686    2559 patch_prober.go:28] interesting pod/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 23 13:46:56.957204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:56.956753    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 23 13:46:57.956749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:57.956713    2559 patch_prober.go:28] interesting pod/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 23 13:46:57.957141 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:57.956764    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 23 13:46:58.960023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:58.959992    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:58.960489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:58.960305    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:46:58.961127 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:46:58.961111    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:47:00.706456 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:00.706422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:00.808978 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:00.808942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:13.473787 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:13.473743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:14.135767 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:14.135729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:14.212077 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:14.212046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:14.272767 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:14.272734    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:18.749865 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:18.749826    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:18.957275 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:18.957247    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"]
Jul 23 13:47:18.957476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:18.957454    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy" containerID="cri-o://55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e" gracePeriod=30
Jul 23 13:47:19.005382 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:19.005310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:19.287832 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:19.287755    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:19.796112 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:19.796079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:19.941696 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:19.941663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:20.071330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:20.071249    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:21.834948 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:21.834914    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:24.214677 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.214651    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:47:24.289697 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289622    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-envoy\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289697 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289679    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-data\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289727    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9txbm\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-kube-api-access-9txbm\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289758    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-certs\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289781    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f074884-4723-4515-8a03-ee79d1debbc5-istio-podinfo\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289818    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-credential-socket\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289846    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f074884-4723-4515-8a03-ee79d1debbc5-istiod-ca-cert\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289872    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-istio-token\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.289982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.289903    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-socket\") pod \"6f074884-4723-4515-8a03-ee79d1debbc5\" (UID: \"6f074884-4723-4515-8a03-ee79d1debbc5\") "
Jul 23 13:47:24.290341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.290063    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:47:24.290341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.290192    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.290467 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.290442    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:47:24.290528 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.290464    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f074884-4723-4515-8a03-ee79d1debbc5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:47:24.290708 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.290683    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:47:24.291043 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.291007    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:47:24.291897 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.291877    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:47:24.292191 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.292159    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f074884-4723-4515-8a03-ee79d1debbc5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:47:24.292488 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.292473    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-kube-api-access-9txbm" (OuterVolumeSpecName: "kube-api-access-9txbm") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "kube-api-access-9txbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:47:24.292619 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.292598    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f074884-4723-4515-8a03-ee79d1debbc5" (UID: "6f074884-4723-4515-8a03-ee79d1debbc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:47:24.390885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390861    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.390885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390882    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.391022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390891    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9txbm\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-kube-api-access-9txbm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.391022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390900    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f074884-4723-4515-8a03-ee79d1debbc5-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.391022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390908    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.391022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390916    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f074884-4723-4515-8a03-ee79d1debbc5-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.391022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390925    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f074884-4723-4515-8a03-ee79d1debbc5-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.391022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.390933    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f074884-4723-4515-8a03-ee79d1debbc5-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:47:24.454280 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.454247    2559 generic.go:356] "Generic (PLEG): container finished" podID="6f074884-4723-4515-8a03-ee79d1debbc5" containerID="55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e" exitCode=0
Jul 23 13:47:24.454402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.454299    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"
Jul 23 13:47:24.454402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.454310    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" event={"ID":"6f074884-4723-4515-8a03-ee79d1debbc5","Type":"ContainerDied","Data":"55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e"}
Jul 23 13:47:24.454402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.454382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg" event={"ID":"6f074884-4723-4515-8a03-ee79d1debbc5","Type":"ContainerDied","Data":"ed7169b34021d6d4a3612eeb8d0108f644a5bc0db0018cdc91d412375d4f1f11"}
Jul 23 13:47:24.454547 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.454410    2559 scope.go:117] "RemoveContainer" containerID="55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e"
Jul 23 13:47:24.464150 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.464135    2559 scope.go:117] "RemoveContainer" containerID="29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"
Jul 23 13:47:24.471993 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.471977    2559 scope.go:117] "RemoveContainer" containerID="55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e"
Jul 23 13:47:24.472251 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:47:24.472232    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e\": container with ID starting with 55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e not found: ID does not exist" containerID="55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e"
Jul 23 13:47:24.472331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.472264    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e"} err="failed to get container status \"55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e\": rpc error: code = NotFound desc = could not find container \"55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e\": container with ID starting with 55e07a896c8c590e79d1fe29c43a1799293c6ca0f49fc6bb2a8fa7906d78f69e not found: ID does not exist"
Jul 23 13:47:24.472331 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.472288    2559 scope.go:117] "RemoveContainer" containerID="29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"
Jul 23 13:47:24.472588 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:47:24.472564    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7\": container with ID starting with 29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7 not found: ID does not exist" containerID="29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"
Jul 23 13:47:24.472633 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.472600    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7"} err="failed to get container status \"29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7\": rpc error: code = NotFound desc = could not find container \"29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7\": container with ID starting with 29fe5e153256b5781c9e8ef45137be1fa6c889328dab3b203ef8438e604b63a7 not found: ID does not exist"
Jul 23 13:47:24.478677 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.478655    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"]
Jul 23 13:47:24.486364 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:24.486343    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-dceaf465-openshift-default-55b9548d87-9prxg"]
Jul 23 13:47:26.092089 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:26.092056    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" path="/var/lib/kubelet/pods/6f074884-4723-4515-8a03-ee79d1debbc5/volumes"
Jul 23 13:47:28.529190 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:28.529158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:28.638045 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:28.638013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:28.737136 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:28.737101    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:28.831459 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:28.831425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:29.350589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:29.350559    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:47:55.388817 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:47:55.388782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:02.237348 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:02.237314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:02.337523 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:02.337485    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:02.438724 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:02.438672    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:02.536804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:02.536696    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:18.535703 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:18.535662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:24.225945 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:24.225908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:24.413913 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:24.413874    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:24.494037 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:24.493954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:34.997314 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:34.997273    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:35.838066 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:35.838030    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:43.972954 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:43.972919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:44.192186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:44.192151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:44.341066 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:44.341019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:45.317160 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:45.317118    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:45.433716 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:45.433671    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:47.414366 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:47.414333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:54.359874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:54.359842    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:48:55.842792 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:48:55.842761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:01.423905 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:01.423871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:01.517333 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:01.517285    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:01.686737 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:01.686653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:02.059065 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.058981    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"]
Jul 23 13:49:02.059561 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059527    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy"
Jul 23 13:49:02.059618 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059564    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy"
Jul 23 13:49:02.059618 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059590    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy"
Jul 23 13:49:02.059618 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059599    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy"
Jul 23 13:49:02.059618 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059610    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy"
Jul 23 13:49:02.059618 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059619    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy"
Jul 23 13:49:02.059773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059705    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy"
Jul 23 13:49:02.059773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059716    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f074884-4723-4515-8a03-ee79d1debbc5" containerName="istio-proxy"
Jul 23 13:49:02.059773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059725    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy"
Jul 23 13:49:02.059773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059733    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy"
Jul 23 13:49:02.059773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.059742    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fb48b1-4e50-4bd2-a4ac-cf5b82c47a83" containerName="istio-proxy"
Jul 23 13:49:02.063302 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.063285    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.066465 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.066448    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ae29a373-openshift-default-dockercfg-qnhtq\""
Jul 23 13:49:02.106804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.106772    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"]
Jul 23 13:49:02.113609 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.113584    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:02.242385 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.242349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:02.248572 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248524    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istiod-ca-cert\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248667 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248582    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-podinfo\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248667 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-certs\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248667 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-credential-socket\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-token\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-data\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lmwr\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-kube-api-access-7lmwr\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248922 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-socket\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.248922 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.248839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-envoy\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350143 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-socket\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-envoy\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istiod-ca-cert\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-podinfo\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-certs\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-credential-socket\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350630 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350334    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-token\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350630 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-data\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350630 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lmwr\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-kube-api-access-7lmwr\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350630 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350602    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-socket\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350827 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350688    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-certs\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350827 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-credential-socket\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.350917 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.350864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-data\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.351113 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.351092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istiod-ca-cert\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.353200 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.353182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-envoy\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.353459 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.353437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-podinfo\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.365206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.365183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lmwr\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-kube-api-access-7lmwr\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.365294 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.365216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-token\") pod \"e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") " pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.374102 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.374078    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:02.518835 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.518811    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"]
Jul 23 13:49:02.520426 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:49:02.520400    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5665cfd2_a7b3_4fe7_b3f3_e2536cf1fd03.slice/crio-b3fe82ea123168899c6091774a9136ed3de00f69c8005f8280982506a6bf502a WatchSource:0}: Error finding container b3fe82ea123168899c6091774a9136ed3de00f69c8005f8280982506a6bf502a: Status 404 returned error can't find the container with id b3fe82ea123168899c6091774a9136ed3de00f69c8005f8280982506a6bf502a
Jul 23 13:49:02.522477 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.522448    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:49:02.522557 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.522524    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:49:02.522606 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.522579    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:49:02.925940 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.925907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" event={"ID":"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03","Type":"ContainerStarted","Data":"483388f51f4df9dea65348d26496df6ae7ba09f82ffbf508f0f655427c47927f"}
Jul 23 13:49:02.926125 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.925968    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" event={"ID":"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03","Type":"ContainerStarted","Data":"b3fe82ea123168899c6091774a9136ed3de00f69c8005f8280982506a6bf502a"}
Jul 23 13:49:02.955741 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:02.955696    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podStartSLOduration=0.955681496 podStartE2EDuration="955.681496ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:49:02.954723468 +0000 UTC m=+1873.330674246" watchObservedRunningTime="2026-07-23 13:49:02.955681496 +0000 UTC m=+1873.331632343"
Jul 23 13:49:03.375172 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:03.375142    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:03.379911 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:03.379883    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:03.930480 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:03.930448    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:03.931399 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:03.931382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:05.245624 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:05.245525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:05.419349 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:05.419314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:05.541968 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:05.541883    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:05.702752 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:05.702719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:06.147853 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:06.147807    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:06.242714 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:06.242683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:09.671646 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:09.671604    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:10.438683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:10.438652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:10.544401 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:10.544369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:10.903002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:10.902972    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:16.808373 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:16.808318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:16.906104 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:16.906064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:18.932583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:18.932459    2559 patch_prober.go:28] interesting pod/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 23 13:49:18.932583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:18.932520    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 23 13:49:19.002059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:19.002024    2559 generic.go:356] "Generic (PLEG): container finished" podID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerID="483388f51f4df9dea65348d26496df6ae7ba09f82ffbf508f0f655427c47927f" exitCode=0
Jul 23 13:49:19.002233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:19.002099    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" event={"ID":"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03","Type":"ContainerDied","Data":"483388f51f4df9dea65348d26496df6ae7ba09f82ffbf508f0f655427c47927f"}
Jul 23 13:49:19.002470 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:19.002457    2559 scope.go:117] "RemoveContainer" containerID="483388f51f4df9dea65348d26496df6ae7ba09f82ffbf508f0f655427c47927f"
Jul 23 13:49:19.003455 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:19.003430    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:49:19.003507 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:19.003491    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:49:19.003559 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:19.003524    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:49:20.010654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:20.010623    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" event={"ID":"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03","Type":"ContainerStarted","Data":"dcfbd7d08bc4a96306c617290f629a402fcc3de75b4ce50f4fad417faab81715"}
Jul 23 13:49:20.374594 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:20.374568    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:20.374807 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:20.374785    2559 patch_prober.go:28] interesting pod/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 23 13:49:20.374862 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:20.374835    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 23 13:49:21.374746 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:21.374713    2559 patch_prober.go:28] interesting pod/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 23 13:49:21.375111 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:21.374774    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 23 13:49:22.374662 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:22.374623    2559 patch_prober.go:28] interesting pod/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 23 13:49:22.374844 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:22.374693    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 23 13:49:23.374895 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:23.374856    2559 patch_prober.go:28] interesting pod/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 23 13:49:23.375255 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:23.374928    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 23 13:49:24.378502 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:24.378472    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:24.378916 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:24.378721    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:24.379400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:24.379382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:27.418794 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:27.418764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:32.281919 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:32.281880    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:32.700723 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:32.700693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:38.615806 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:38.615764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:38.716642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:38.716610    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:38.890619 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:38.890522    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"]
Jul 23 13:49:38.890827 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:38.890796    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" containerID="cri-o://dcfbd7d08bc4a96306c617290f629a402fcc3de75b4ce50f4fad417faab81715" gracePeriod=30
Jul 23 13:49:38.906464 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:38.906440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:39.380798 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:39.380767    2559 patch_prober.go:28] interesting pod/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 23 13:49:39.380967 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:39.380829    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 13:49:40.189429 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:40.189395    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:40.351343 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:40.351292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:40.549899 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:40.549790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:40.647648 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:40.647614    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:44.123766 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.123732    2559 generic.go:356] "Generic (PLEG): container finished" podID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerID="dcfbd7d08bc4a96306c617290f629a402fcc3de75b4ce50f4fad417faab81715" exitCode=0
Jul 23 13:49:44.124165 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.123803    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" event={"ID":"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03","Type":"ContainerDied","Data":"dcfbd7d08bc4a96306c617290f629a402fcc3de75b4ce50f4fad417faab81715"}
Jul 23 13:49:44.124165 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.123868    2559 scope.go:117] "RemoveContainer" containerID="483388f51f4df9dea65348d26496df6ae7ba09f82ffbf508f0f655427c47927f"
Jul 23 13:49:44.157460 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.157408    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:44.319019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.318991    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lmwr\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-kube-api-access-7lmwr\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319035    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-envoy\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319063    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istiod-ca-cert\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319082    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-socket\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319102    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-podinfo\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319134    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-token\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-certs\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319197    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-data\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319220    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-credential-socket\") pod \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\" (UID: \"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03\") "
Jul 23 13:49:44.319642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319413    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:49:44.319821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319681    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:49:44.319821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319704    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:49:44.320059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.319769    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:49:44.321087 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.321049    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-data" (OuterVolumeSpecName: "istio-data") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:49:44.321524 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.321502    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:49:44.321631 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.321576    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-kube-api-access-7lmwr" (OuterVolumeSpecName: "kube-api-access-7lmwr") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "kube-api-access-7lmwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:49:44.321631 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.321601    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:49:44.321729 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.321647    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-token" (OuterVolumeSpecName: "istio-token") pod "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" (UID: "5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420645    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420670    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420680    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420687    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420697    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lmwr\" (UniqueName: \"kubernetes.io/projected/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-kube-api-access-7lmwr\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420706    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420714    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.420719 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420722    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:44.421056 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:44.420729    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:49:45.130479 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:45.130448    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"
Jul 23 13:49:45.130959 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:45.130484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn" event={"ID":"5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03","Type":"ContainerDied","Data":"b3fe82ea123168899c6091774a9136ed3de00f69c8005f8280982506a6bf502a"}
Jul 23 13:49:45.130959 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:45.130557    2559 scope.go:117] "RemoveContainer" containerID="dcfbd7d08bc4a96306c617290f629a402fcc3de75b4ce50f4fad417faab81715"
Jul 23 13:49:45.153121 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:45.153092    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"]
Jul 23 13:49:45.161410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:45.161386    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae29a373-openshift-default-6db9cdcfff-kq4bn"]
Jul 23 13:49:46.091780 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:46.091744    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" path="/var/lib/kubelet/pods/5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03/volumes"
Jul 23 13:49:46.485121 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:46.485033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:46.585866 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:46.585830    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:46.686020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:46.685987    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:46.871072 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:46.871042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:52.650021 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:52.649968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:52.749147 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:52.749114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:49:59.731753 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:49:59.731722    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:00.048615 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.048512    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"]
Jul 23 13:50:00.049033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049008    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy"
Jul 23 13:50:00.049033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049035    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7579573-2231-4faf-842e-5c97bbec47e3" containerName="istio-proxy"
Jul 23 13:50:00.049225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049057    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy"
Jul 23 13:50:00.049225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049067    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy"
Jul 23 13:50:00.049225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049077    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy"
Jul 23 13:50:00.049225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049082    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy"
Jul 23 13:50:00.049225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.049170    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy"
Jul 23 13:50:00.052390 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.052374    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055302 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055278    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9p6g\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-kube-api-access-d9p6g\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055404 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-socket\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055404 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055343    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cca0432-3155-491a-ac51-d645251f0058-istiod-ca-cert\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055404 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-envoy\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-certs\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-data\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-credential-socket\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055743 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-istio-token\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.055743 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.055644    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cca0432-3155-491a-ac51-d645251f0058-istio-podinfo\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.056664 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.056646    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5f52d341-openshift-default-dockercfg-jf4bv\""
Jul 23 13:50:00.074583 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.074556    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"]
Jul 23 13:50:00.117650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.117616    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:00.156620 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.156579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9p6g\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-kube-api-access-d9p6g\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.156821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.156634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-socket\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.156821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.156664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cca0432-3155-491a-ac51-d645251f0058-istiod-ca-cert\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.156821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.156688    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-envoy\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.156821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.156729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-certs\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.156821 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.156770    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-data\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.157416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.157364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-certs\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.157571 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.157451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-data\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.157735 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.157705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-credential-socket\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.157859 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.157758    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-istio-token\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.157859 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.157806    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cca0432-3155-491a-ac51-d645251f0058-istio-podinfo\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.157976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.157873    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cca0432-3155-491a-ac51-d645251f0058-istiod-ca-cert\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.158428 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.158406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-credential-socket\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.158851 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.158778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-socket\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.161606 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.160971    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-envoy\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.161606 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.161432    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cca0432-3155-491a-ac51-d645251f0058-istio-podinfo\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.169377 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.169352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-istio-token\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.172737 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.172715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9p6g\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-kube-api-access-d9p6g\") pod \"e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") " pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.363086 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.363046    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:00.483236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.483201    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:00.515342 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.515316    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"]
Jul 23 13:50:00.515934 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:50:00.515909    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cca0432_3155_491a_ac51_d645251f0058.slice/crio-926b070b42c35635a015126fb13f2f6b94fe56d2aa813374a972577908759ca4 WatchSource:0}: Error finding container 926b070b42c35635a015126fb13f2f6b94fe56d2aa813374a972577908759ca4: Status 404 returned error can't find the container with id 926b070b42c35635a015126fb13f2f6b94fe56d2aa813374a972577908759ca4
Jul 23 13:50:00.518040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.518008    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:50:00.518129 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.518072    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:50:00.518129 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:00.518107    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:50:01.209414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:01.209375    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" event={"ID":"7cca0432-3155-491a-ac51-d645251f0058","Type":"ContainerStarted","Data":"f5e8996d3081aaea543f51386dbfacb549993f2978343762a3022c2a4460b0d1"}
Jul 23 13:50:01.209842 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:01.209436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" event={"ID":"7cca0432-3155-491a-ac51-d645251f0058","Type":"ContainerStarted","Data":"926b070b42c35635a015126fb13f2f6b94fe56d2aa813374a972577908759ca4"}
Jul 23 13:50:01.233547 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:01.233489    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podStartSLOduration=1.233473296 podStartE2EDuration="1.233473296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:50:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:50:01.231854254 +0000 UTC m=+1931.607805033" watchObservedRunningTime="2026-07-23 13:50:01.233473296 +0000 UTC m=+1931.609424138"
Jul 23 13:50:01.363363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:01.363320    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:01.364970 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:01.364945    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 23 13:50:01.365105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:01.364994    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 23 13:50:02.367603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:02.367569    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:03.219991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:03.219955    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:03.220875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:03.220858    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:03.237575 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:03.237519    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:03.445802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:03.445771    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:03.714071 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:03.714022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:03.832410 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:03.832380    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:04.214505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:04.214467    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:04.315847 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:04.315817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:07.776423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:07.776385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:18.221556 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.221502    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 23 13:50:18.221970 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.221586    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 23 13:50:18.296031 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.295982    2559 generic.go:356] "Generic (PLEG): container finished" podID="7cca0432-3155-491a-ac51-d645251f0058" containerID="f5e8996d3081aaea543f51386dbfacb549993f2978343762a3022c2a4460b0d1" exitCode=0
Jul 23 13:50:18.296213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.296050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" event={"ID":"7cca0432-3155-491a-ac51-d645251f0058","Type":"ContainerDied","Data":"f5e8996d3081aaea543f51386dbfacb549993f2978343762a3022c2a4460b0d1"}
Jul 23 13:50:18.296597 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.296578    2559 scope.go:117] "RemoveContainer" containerID="f5e8996d3081aaea543f51386dbfacb549993f2978343762a3022c2a4460b0d1"
Jul 23 13:50:18.297718 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.297684    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:50:18.297809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.297776    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:50:18.297848 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:18.297824    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:50:19.306105 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:19.306065    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" event={"ID":"7cca0432-3155-491a-ac51-d645251f0058","Type":"ContainerStarted","Data":"147e2496a977e9f5b066f3f594353a60541aa52d1a75633f8ff02c258c7f11cc"}
Jul 23 13:50:19.364060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:19.364021    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:19.364432 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:19.364211    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 23 13:50:19.364432 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:19.364275    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 23 13:50:20.363942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:20.363899    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 23 13:50:20.364374 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:20.363984    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 23 13:50:21.363590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:21.363525    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 23 13:50:21.363801 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:21.363614    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 23 13:50:22.363585 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:22.363524    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 23 13:50:22.364052 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:22.363621    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 23 13:50:23.367948 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:23.367918    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:23.368452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:23.368430    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:23.368930 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:23.368910    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:50:25.946888 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.946852    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"]
Jul 23 13:50:25.947349 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.947291    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5665cfd2-a7b3-4fe7-b3f3-e2536cf1fd03" containerName="istio-proxy"
Jul 23 13:50:25.950291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.950271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:25.953882 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.953855    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5f52d341\"/\"default-dockercfg-5xwsg\""
Jul 23 13:50:25.954016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.953881    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5f52d341\"/\"openshift-service-ca.crt\""
Jul 23 13:50:25.955039 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.955008    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5f52d341\"/\"e2e-lifecycle-model-5f52d341-kserve-self-signed-certs\""
Jul 23 13:50:25.955139 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.955083    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5f52d341\"/\"kube-root-ca.crt\""
Jul 23 13:50:25.963380 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.963355    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"]
Jul 23 13:50:25.992277 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:25.992242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:50:26.106444 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.106415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q7z4\" (UniqueName: \"kubernetes.io/projected/57f76def-e388-44dd-8f50-e70ffd257676-kube-api-access-9q7z4\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.106641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.106472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-dshm\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.106641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.106578    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-tmp-dir\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.106641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.106632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-home\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.106750 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.106700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-model-cache\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.106750 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.106726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f76def-e388-44dd-8f50-e70ffd257676-tls-certs\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208135 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-dshm\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208135 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-tmp-dir\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208135 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-home\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-model-cache\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208181    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f76def-e388-44dd-8f50-e70ffd257676-tls-certs\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208452 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q7z4\" (UniqueName: \"kubernetes.io/projected/57f76def-e388-44dd-8f50-e70ffd257676-kube-api-access-9q7z4\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208567    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-tmp-dir\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208717 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-home\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.208793 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.208776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-model-cache\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.210607 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.210582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-dshm\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.210872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.210857    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f76def-e388-44dd-8f50-e70ffd257676-tls-certs\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.218682 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.218655    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q7z4\" (UniqueName: \"kubernetes.io/projected/57f76def-e388-44dd-8f50-e70ffd257676-kube-api-access-9q7z4\") pod \"e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") " pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.260400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.260362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:26.417297 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:26.417261    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"]
Jul 23 13:50:26.420911 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:50:26.420878    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57f76def_e388_44dd_8f50_e70ffd257676.slice/crio-45a5b4ffc03237d2624b576a3e83dfcea7441730a195bbd2c10f931304ca0706 WatchSource:0}: Error finding container 45a5b4ffc03237d2624b576a3e83dfcea7441730a195bbd2c10f931304ca0706: Status 404 returned error can't find the container with id 45a5b4ffc03237d2624b576a3e83dfcea7441730a195bbd2c10f931304ca0706
Jul 23 13:50:27.348485 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:27.348448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm" event={"ID":"57f76def-e388-44dd-8f50-e70ffd257676","Type":"ContainerStarted","Data":"5bc8221ade1eede0e11bde3a4dcd02343fc782f3b3f2d91d52704f89298568be"}
Jul 23 13:50:27.348485 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:27.348488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm" event={"ID":"57f76def-e388-44dd-8f50-e70ffd257676","Type":"ContainerStarted","Data":"45a5b4ffc03237d2624b576a3e83dfcea7441730a195bbd2c10f931304ca0706"}
Jul 23 13:50:27.368234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:27.368177    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm" podStartSLOduration=2.368160402 podStartE2EDuration="2.368160402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:50:27.36663833 +0000 UTC m=+1957.742589109" watchObservedRunningTime="2026-07-23 13:50:27.368160402 +0000 UTC m=+1957.744111180"
Jul 23 13:50:36.261150 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:36.261109    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:36.261684 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:36.261166    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:36.277786 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:36.277758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:50:36.405437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:50:36.405411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:51:12.536918 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:12.536847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:51:17.112161 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:17.112125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:51:19.529886 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:19.529854    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:51:24.499845 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:24.499806    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"]
Jul 23 13:51:24.551888 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:24.551837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:51:39.520393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.520336    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm" podUID="57f76def-e388-44dd-8f50-e70ffd257676" containerName="main" containerID="cri-o://5bc8221ade1eede0e11bde3a4dcd02343fc782f3b3f2d91d52704f89298568be" gracePeriod=45
Jul 23 13:51:39.703673 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.703642    2559 generic.go:356] "Generic (PLEG): container finished" podID="57f76def-e388-44dd-8f50-e70ffd257676" containerID="5bc8221ade1eede0e11bde3a4dcd02343fc782f3b3f2d91d52704f89298568be" exitCode=0
Jul 23 13:51:39.703827 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.703685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm" event={"ID":"57f76def-e388-44dd-8f50-e70ffd257676","Type":"ContainerDied","Data":"5bc8221ade1eede0e11bde3a4dcd02343fc782f3b3f2d91d52704f89298568be"}
Jul 23 13:51:39.781397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.781330    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:51:39.875323 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875296    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-model-cache\") pod \"57f76def-e388-44dd-8f50-e70ffd257676\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") "
Jul 23 13:51:39.875476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875347    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-tmp-dir\") pod \"57f76def-e388-44dd-8f50-e70ffd257676\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") "
Jul 23 13:51:39.875476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875399    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q7z4\" (UniqueName: \"kubernetes.io/projected/57f76def-e388-44dd-8f50-e70ffd257676-kube-api-access-9q7z4\") pod \"57f76def-e388-44dd-8f50-e70ffd257676\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") "
Jul 23 13:51:39.875476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875421    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-dshm\") pod \"57f76def-e388-44dd-8f50-e70ffd257676\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") "
Jul 23 13:51:39.875476 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875444    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f76def-e388-44dd-8f50-e70ffd257676-tls-certs\") pod \"57f76def-e388-44dd-8f50-e70ffd257676\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") "
Jul 23 13:51:39.875657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875484    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-home\") pod \"57f76def-e388-44dd-8f50-e70ffd257676\" (UID: \"57f76def-e388-44dd-8f50-e70ffd257676\") "
Jul 23 13:51:39.875657 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875566    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-model-cache" (OuterVolumeSpecName: "model-cache") pod "57f76def-e388-44dd-8f50-e70ffd257676" (UID: "57f76def-e388-44dd-8f50-e70ffd257676"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:51:39.875742 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875698    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57f76def-e388-44dd-8f50-e70ffd257676" (UID: "57f76def-e388-44dd-8f50-e70ffd257676"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:51:39.875804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875777    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-home" (OuterVolumeSpecName: "home") pod "57f76def-e388-44dd-8f50-e70ffd257676" (UID: "57f76def-e388-44dd-8f50-e70ffd257676"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:51:39.875901 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875816    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:51:39.875901 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.875832    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:51:39.877809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.877782    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-dshm" (OuterVolumeSpecName: "dshm") pod "57f76def-e388-44dd-8f50-e70ffd257676" (UID: "57f76def-e388-44dd-8f50-e70ffd257676"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:51:39.877809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.877801    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f76def-e388-44dd-8f50-e70ffd257676-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57f76def-e388-44dd-8f50-e70ffd257676" (UID: "57f76def-e388-44dd-8f50-e70ffd257676"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:51:39.877962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.877830    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f76def-e388-44dd-8f50-e70ffd257676-kube-api-access-9q7z4" (OuterVolumeSpecName: "kube-api-access-9q7z4") pod "57f76def-e388-44dd-8f50-e70ffd257676" (UID: "57f76def-e388-44dd-8f50-e70ffd257676"). InnerVolumeSpecName "kube-api-access-9q7z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:51:39.976394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.976363    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q7z4\" (UniqueName: \"kubernetes.io/projected/57f76def-e388-44dd-8f50-e70ffd257676-kube-api-access-9q7z4\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:51:39.976394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.976388    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:51:39.976394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.976398    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f76def-e388-44dd-8f50-e70ffd257676-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:51:39.976636 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:39.976407    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f76def-e388-44dd-8f50-e70ffd257676-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:51:40.709717 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:40.709687    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"
Jul 23 13:51:40.710195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:40.709689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm" event={"ID":"57f76def-e388-44dd-8f50-e70ffd257676","Type":"ContainerDied","Data":"45a5b4ffc03237d2624b576a3e83dfcea7441730a195bbd2c10f931304ca0706"}
Jul 23 13:51:40.710195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:40.709839    2559 scope.go:117] "RemoveContainer" containerID="5bc8221ade1eede0e11bde3a4dcd02343fc782f3b3f2d91d52704f89298568be"
Jul 23 13:51:40.731020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:40.730996    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"]
Jul 23 13:51:40.738670 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:40.738648    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5f52d341/e2e-lifecycle-model-5f52d341-kserve-54d986667f-n4pvm"]
Jul 23 13:51:42.092033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:42.091999    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f76def-e388-44dd-8f50-e70ffd257676" path="/var/lib/kubelet/pods/57f76def-e388-44dd-8f50-e70ffd257676/volumes"
Jul 23 13:51:50.503833 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:50.503799    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:51:50.601518 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:51:50.601490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:01.349341 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:01.349299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:02.338098 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:02.338061    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:03.533638 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:03.533609    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:03.876461 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:03.876425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:07.476050 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:07.476013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:07.575961 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:07.575929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:07.753968 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:07.753883    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"]
Jul 23 13:52:07.754114 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:07.754087    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" containerID="cri-o://147e2496a977e9f5b066f3f594353a60541aa52d1a75633f8ff02c258c7f11cc" gracePeriod=30
Jul 23 13:52:07.776311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:07.776284    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:08.045782 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:08.045684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:08.370866 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:08.370831    2559 patch_prober.go:28] interesting pod/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 13:52:08.371022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:08.370893    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 13:52:09.232116 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:09.232068    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:09.423820 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:09.423787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:10.273515 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:10.273474    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:10.370874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:10.370844    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:10.470921 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:10.470886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:11.934009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:11.933933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:12.874915 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:12.874884    2559 generic.go:356] "Generic (PLEG): container finished" podID="7cca0432-3155-491a-ac51-d645251f0058" containerID="147e2496a977e9f5b066f3f594353a60541aa52d1a75633f8ff02c258c7f11cc" exitCode=0
Jul 23 13:52:12.875051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:12.874941    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" event={"ID":"7cca0432-3155-491a-ac51-d645251f0058","Type":"ContainerDied","Data":"147e2496a977e9f5b066f3f594353a60541aa52d1a75633f8ff02c258c7f11cc"}
Jul 23 13:52:12.875051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:12.874990    2559 scope.go:117] "RemoveContainer" containerID="f5e8996d3081aaea543f51386dbfacb549993f2978343762a3022c2a4460b0d1"
Jul 23 13:52:13.009078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.009053    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:52:13.175844 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.175804    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cca0432-3155-491a-ac51-d645251f0058-istio-podinfo\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176006 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.175862    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-credential-socket\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176006 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.175898    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9p6g\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-kube-api-access-d9p6g\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176006 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.175932    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cca0432-3155-491a-ac51-d645251f0058-istiod-ca-cert\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176006 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.175972    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-envoy\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176018    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-data\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176060    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-istio-token\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176106    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-certs\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176148    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-socket\") pod \"7cca0432-3155-491a-ac51-d645251f0058\" (UID: \"7cca0432-3155-491a-ac51-d645251f0058\") "
Jul 23 13:52:13.176436 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176244    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:52:13.176436 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176376    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cca0432-3155-491a-ac51-d645251f0058-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:52:13.176582 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176464    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cca0432-3155-491a-ac51-d645251f0058-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.176582 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176484    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.176695 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176626    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:52:13.176787 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.176769    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:52:13.177134 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.177104    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:52:13.178336 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.178299    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cca0432-3155-491a-ac51-d645251f0058-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:52:13.178447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.178374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:52:13.178783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.178759    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:52:13.178783 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.178772    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-kube-api-access-d9p6g" (OuterVolumeSpecName: "kube-api-access-d9p6g") pod "7cca0432-3155-491a-ac51-d645251f0058" (UID: "7cca0432-3155-491a-ac51-d645251f0058"). InnerVolumeSpecName "kube-api-access-d9p6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:52:13.277344 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277317    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.277344 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277341    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.277492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277350    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.277492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277358    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cca0432-3155-491a-ac51-d645251f0058-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.277492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277367    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9p6g\" (UniqueName: \"kubernetes.io/projected/7cca0432-3155-491a-ac51-d645251f0058-kube-api-access-d9p6g\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.277492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277377    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.277492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.277385    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cca0432-3155-491a-ac51-d645251f0058-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:52:13.881194 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.881166    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"
Jul 23 13:52:13.881194 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.881192    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf" event={"ID":"7cca0432-3155-491a-ac51-d645251f0058","Type":"ContainerDied","Data":"926b070b42c35635a015126fb13f2f6b94fe56d2aa813374a972577908759ca4"}
Jul 23 13:52:13.881468 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.881245    2559 scope.go:117] "RemoveContainer" containerID="147e2496a977e9f5b066f3f594353a60541aa52d1a75633f8ff02c258c7f11cc"
Jul 23 13:52:13.907044 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.907018    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"]
Jul 23 13:52:13.919412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:13.919388    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5f52d341-openshift-default-9957d7c5-lfbwf"]
Jul 23 13:52:14.092976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:14.092946    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cca0432-3155-491a-ac51-d645251f0058" path="/var/lib/kubelet/pods/7cca0432-3155-491a-ac51-d645251f0058/volumes"
Jul 23 13:52:17.880181 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:17.880144    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:17.979482 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:17.979452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:18.583007 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:18.582971    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:25.239590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:25.239559    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:25.312310 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:25.312280    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:25.416020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:25.415983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:25.539119 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:25.539036    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:34.684339 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:34.684305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:34.786419 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:34.786387    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:36.063505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:36.063457    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:36.441424 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:36.441385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:52:57.508524 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:52:57.508458    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:03.986431 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:03.986388    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:04.085286 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.085248    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:04.393874 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.393840    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:04.714519 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714446    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"]
Jul 23 13:53:04.714834 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714821    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f76def-e388-44dd-8f50-e70ffd257676" containerName="main"
Jul 23 13:53:04.714885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714836    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f76def-e388-44dd-8f50-e70ffd257676" containerName="main"
Jul 23 13:53:04.714885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714846    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy"
Jul 23 13:53:04.714885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714852    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy"
Jul 23 13:53:04.714885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714860    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy"
Jul 23 13:53:04.714885 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714867    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy"
Jul 23 13:53:04.715062 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714923    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy"
Jul 23 13:53:04.715062 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.714932    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f76def-e388-44dd-8f50-e70ffd257676" containerName="main"
Jul 23 13:53:04.722100 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.719439    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.723195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.723081    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-918c39-openshift-default-dockercfg-vn2xf\""
Jul 23 13:53:04.733603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.733575    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"]
Jul 23 13:53:04.751462 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.751423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:04.813417 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081b254f-d274-4430-a3e9-69fc2af179b9-istio-podinfo\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-socket\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-istio-token\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqrgk\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-kube-api-access-sqrgk\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813528    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081b254f-d274-4430-a3e9-69fc2af179b9-istiod-ca-cert\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-data\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813815 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-certs\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813815 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-credential-socket\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.813815 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.813682    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-envoy\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.848321 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.848286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:04.914312 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914278    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-credential-socket\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914312 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-envoy\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081b254f-d274-4430-a3e9-69fc2af179b9-istio-podinfo\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-socket\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-istio-token\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqrgk\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-kube-api-access-sqrgk\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081b254f-d274-4430-a3e9-69fc2af179b9-istiod-ca-cert\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-data\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.914614 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-certs\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.915009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914829    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-socket\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.915009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.914955    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-certs\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.915115 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.915007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-data\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.915115 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.915061    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-credential-socket\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.915447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.915425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081b254f-d274-4430-a3e9-69fc2af179b9-istiod-ca-cert\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.916812 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.916790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-envoy\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.917085 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.917064    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081b254f-d274-4430-a3e9-69fc2af179b9-istio-podinfo\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.925162 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.925129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqrgk\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-kube-api-access-sqrgk\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:04.925162 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:04.925151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-istio-token\") pod \"e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") " pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:05.031943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:05.031905    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:05.176121 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:05.176092    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"]
Jul 23 13:53:05.177636 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:53:05.177599    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081b254f_d274_4430_a3e9_69fc2af179b9.slice/crio-2e69769f1da363581ffdad435e86432970b7ae289b332307b03121e7c4d887ff WatchSource:0}: Error finding container 2e69769f1da363581ffdad435e86432970b7ae289b332307b03121e7c4d887ff: Status 404 returned error can't find the container with id 2e69769f1da363581ffdad435e86432970b7ae289b332307b03121e7c4d887ff
Jul 23 13:53:05.180148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:05.180105    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:05.180280 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:05.180220    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:05.180280 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:05.180263    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:06.127511 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:06.127479    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" event={"ID":"081b254f-d274-4430-a3e9-69fc2af179b9","Type":"ContainerStarted","Data":"372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"}
Jul 23 13:53:06.127889 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:06.127522    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" event={"ID":"081b254f-d274-4430-a3e9-69fc2af179b9","Type":"ContainerStarted","Data":"2e69769f1da363581ffdad435e86432970b7ae289b332307b03121e7c4d887ff"}
Jul 23 13:53:06.151934 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:06.151890    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" podStartSLOduration=2.151874859 podStartE2EDuration="2.151874859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:53:06.14972711 +0000 UTC m=+2116.525677914" watchObservedRunningTime="2026-07-23 13:53:06.151874859 +0000 UTC m=+2116.527825637"
Jul 23 13:53:07.032726 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:07.032693    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:07.037427 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:07.037402    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:07.133255 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:07.133229    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:07.134170 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:07.134151    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:08.132711 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:08.132675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:08.229865 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:08.229827    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:08.384100 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:08.384012    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:08.675731 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:08.675647    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:08.957720 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:08.957648    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:09.056280 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:09.056248    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:12.712234 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:12.712201    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:13.057617 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.057503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:13.391976 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.391945    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"]
Jul 23 13:53:13.392465 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.392447    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cca0432-3155-491a-ac51-d645251f0058" containerName="istio-proxy"
Jul 23 13:53:13.395774 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.395737    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.402766 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.402744    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8ecfce-openshift-default-dockercfg-crzqw\""
Jul 23 13:53:13.411361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.411335    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"]
Jul 23 13:53:13.461495 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.461457    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:13.493946 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.493908    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-certs\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.493956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70de1af2-153e-40e9-8b22-5bb46e94637d-istiod-ca-cert\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.493994    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-data\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494128 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.494106    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-credential-socket\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494306 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.494157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-socket\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494306 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.494195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-envoy\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494306 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.494221    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-podinfo\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494306 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.494271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9hqk\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-kube-api-access-h9hqk\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.494518 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.494351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-token\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.595797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.595762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-certs\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596004 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.595820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70de1af2-153e-40e9-8b22-5bb46e94637d-istiod-ca-cert\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596004 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.595855    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-data\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596004 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.595909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-credential-socket\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596004 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.595944    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-socket\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596203    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-certs\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596218    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-credential-socket\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-envoy\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-podinfo\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-data\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596347 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9hqk\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-kube-api-access-h9hqk\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596593 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-token\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.596658 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.596589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-socket\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.597393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.597343    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70de1af2-153e-40e9-8b22-5bb46e94637d-istiod-ca-cert\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.599456 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.599430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-envoy\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.599613 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.599586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-podinfo\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.605599 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.605576    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-token\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.605888 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.605869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9hqk\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-kube-api-access-h9hqk\") pod \"e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") " pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.707576 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.707479    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:13.786561 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.785967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:13.893355 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.893249    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"]
Jul 23 13:53:13.910460 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:53:13.910428    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70de1af2_153e_40e9_8b22_5bb46e94637d.slice/crio-663ccbf5a5604714eea7202c35b05028dfa6fdabaf7af696bb96b760c03f3cd0 WatchSource:0}: Error finding container 663ccbf5a5604714eea7202c35b05028dfa6fdabaf7af696bb96b760c03f3cd0: Status 404 returned error can't find the container with id 663ccbf5a5604714eea7202c35b05028dfa6fdabaf7af696bb96b760c03f3cd0
Jul 23 13:53:13.912696 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.912664    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:13.912793 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.912750    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:13.912834 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:13.912792    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:14.167642 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:14.167607    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" event={"ID":"70de1af2-153e-40e9-8b22-5bb46e94637d","Type":"ContainerStarted","Data":"d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"}
Jul 23 13:53:14.167846 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:14.167664    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" event={"ID":"70de1af2-153e-40e9-8b22-5bb46e94637d","Type":"ContainerStarted","Data":"663ccbf5a5604714eea7202c35b05028dfa6fdabaf7af696bb96b760c03f3cd0"}
Jul 23 13:53:14.197394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:14.197331    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podStartSLOduration=1.197305169 podStartE2EDuration="1.197305169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:53:14.194323254 +0000 UTC m=+2124.570274033" watchObservedRunningTime="2026-07-23 13:53:14.197305169 +0000 UTC m=+2124.573255947"
Jul 23 13:53:14.708350 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:14.708307    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:14.713773 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:14.713740    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:15.172902 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:15.172870    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:15.173804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:15.173783    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:16.263938 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:16.263907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:16.373089 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:16.373007    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:16.461566 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:16.461461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:16.572938 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:16.572857    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:17.096320 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:17.096277    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:17.197873 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:17.197834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:21.158271 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.158233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:21.206284 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.206255    2559 generic.go:356] "Generic (PLEG): container finished" podID="081b254f-d274-4430-a3e9-69fc2af179b9" containerID="372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194" exitCode=0
Jul 23 13:53:21.206445 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.206333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" event={"ID":"081b254f-d274-4430-a3e9-69fc2af179b9","Type":"ContainerDied","Data":"372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"}
Jul 23 13:53:21.206835 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.206819    2559 scope.go:117] "RemoveContainer" containerID="372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"
Jul 23 13:53:21.207819 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.207795    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:21.207882 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.207862    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:21.207930 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:21.207894    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:22.133416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:22.133379    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:22.214077 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:22.214040    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" event={"ID":"081b254f-d274-4430-a3e9-69fc2af179b9","Type":"ContainerStarted","Data":"633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e"}
Jul 23 13:53:23.032510 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:23.032468    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:23.033677 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:23.033644    2559 patch_prober.go:28] interesting pod/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 23 13:53:23.033809 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:23.033704    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 23 13:53:24.033254 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:24.033212    2559 patch_prober.go:28] interesting pod/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 23 13:53:24.033675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:24.033277    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 23 13:53:25.033364 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:25.033323    2559 patch_prober.go:28] interesting pod/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 23 13:53:25.033811 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:25.033394    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 23 13:53:26.033091 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:26.033044    2559 patch_prober.go:28] interesting pod/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 23 13:53:26.033279 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:26.033116    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 23 13:53:27.036847 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:27.036811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:27.037276 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:27.037114    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:27.037712 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:27.037693    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:53:29.471319 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.471282    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"]
Jul 23 13:53:29.474407 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.474387    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.478762 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.478741    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-918c39\"/\"kube-root-ca.crt\""
Jul 23 13:53:29.479939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.479918    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-918c39\"/\"openshift-service-ca.crt\""
Jul 23 13:53:29.480061 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.480012    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-918c39\"/\"default-dockercfg-cqdwv\""
Jul 23 13:53:29.480129 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.480051    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-918c39\"/\"test-model-918c39-kserve-self-signed-certs\""
Jul 23 13:53:29.497257 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.497235    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"]
Jul 23 13:53:29.536487 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.536457    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:29.567603 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.567571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tmp-dir\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.567779 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.567649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tls-certs\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.567779 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.567684    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7458x\" (UniqueName: \"kubernetes.io/projected/09f61fb2-9be1-4f12-a25e-6fcb4b118244-kube-api-access-7458x\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.567779 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.567731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-dshm\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.567779 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.567758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-model-cache\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.567934 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.567780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-home\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.668802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.668759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tmp-dir\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.668877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tls-certs\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.668909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7458x\" (UniqueName: \"kubernetes.io/projected/09f61fb2-9be1-4f12-a25e-6fcb4b118244-kube-api-access-7458x\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.668960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-dshm\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669002 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.668999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-model-cache\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.669046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-home\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.669226    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tmp-dir\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.669441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-home\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.669466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.669457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-model-cache\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.671596 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.671568    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-dshm\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.671755 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.671737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tls-certs\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.686640 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.686607    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7458x\" (UniqueName: \"kubernetes.io/projected/09f61fb2-9be1-4f12-a25e-6fcb4b118244-kube-api-access-7458x\") pod \"test-model-918c39-kserve-6b848d58d7-cvcb7\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") " pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.784826 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.784725    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:29.937249 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:29.937220    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"]
Jul 23 13:53:29.939961 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:53:29.939933    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09f61fb2_9be1_4f12_a25e_6fcb4b118244.slice/crio-871bbcef9543745589f8a1cba2e4017be0cb1c8c7c839ca330ab3464e661338c WatchSource:0}: Error finding container 871bbcef9543745589f8a1cba2e4017be0cb1c8c7c839ca330ab3464e661338c: Status 404 returned error can't find the container with id 871bbcef9543745589f8a1cba2e4017be0cb1c8c7c839ca330ab3464e661338c
Jul 23 13:53:30.018845 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:30.018814    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:30.209932 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:30.209884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:30.258948 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:30.258905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7" event={"ID":"09f61fb2-9be1-4f12-a25e-6fcb4b118244","Type":"ContainerStarted","Data":"2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154"}
Jul 23 13:53:30.258948 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:30.258954    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7" event={"ID":"09f61fb2-9be1-4f12-a25e-6fcb4b118244","Type":"ContainerStarted","Data":"871bbcef9543745589f8a1cba2e4017be0cb1c8c7c839ca330ab3464e661338c"}
Jul 23 13:53:30.287026 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:30.286966    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7" podStartSLOduration=1.286949163 podStartE2EDuration="1.286949163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:53:30.285794788 +0000 UTC m=+2140.661745568" watchObservedRunningTime="2026-07-23 13:53:30.286949163 +0000 UTC m=+2140.662899941"
Jul 23 13:53:31.214705 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:31.214664    2559 patch_prober.go:28] interesting pod/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 13:53:31.215135 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:31.214725    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 13:53:31.536801 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:31.536716    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:31.610159 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:31.610127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:32.271369 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:32.271291    2559 generic.go:356] "Generic (PLEG): container finished" podID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerID="d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a" exitCode=0
Jul 23 13:53:32.271369 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:32.271359    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" event={"ID":"70de1af2-153e-40e9-8b22-5bb46e94637d","Type":"ContainerDied","Data":"d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"}
Jul 23 13:53:32.271814 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:32.271750    2559 scope.go:117] "RemoveContainer" containerID="d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"
Jul 23 13:53:32.272797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:32.272770    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:32.272907 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:32.272861    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:32.272973 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:32.272918    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:53:33.278471 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:33.278438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" event={"ID":"70de1af2-153e-40e9-8b22-5bb46e94637d","Type":"ContainerStarted","Data":"c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f"}
Jul 23 13:53:33.707741 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:33.707706    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:33.708041 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:33.708018    2559 patch_prober.go:28] interesting pod/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 23 13:53:33.708160 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:33.708073    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 23 13:53:34.708280 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:34.708244    2559 patch_prober.go:28] interesting pod/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 23 13:53:34.708701 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:34.708304    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 23 13:53:35.708357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:35.708314    2559 patch_prober.go:28] interesting pod/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 23 13:53:35.708767 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:35.708385    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 23 13:53:36.709050 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:36.709004    2559 patch_prober.go:28] interesting pod/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 23 13:53:36.709574 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:36.709093    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 23 13:53:37.711226 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:37.711192    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:37.711678 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:37.711551    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:37.712158 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:37.712144    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:53:38.112313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:38.112264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:38.211745 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:38.211706    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:53:39.785204 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:39.785172    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:39.785625 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:39.785229    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:39.797664 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:39.797638    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:40.330928 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:40.330896    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:53:52.494059 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:53:52.493974    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:54:23.649602 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.649570    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"]
Jul 23 13:54:23.652662 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.652645    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.656466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.656444    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8ecfce\"/\"default-dockercfg-2bfn7\""
Jul 23 13:54:23.656617 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.656444    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8ecfce\"/\"openshift-service-ca.crt\""
Jul 23 13:54:23.656617 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.656444    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8ecfce\"/\"kube-root-ca.crt\""
Jul 23 13:54:23.656617 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.656449    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8ecfce\"/\"test-model-8ecfce-kserve-self-signed-certs\""
Jul 23 13:54:23.663181 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.663161    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"]
Jul 23 13:54:23.761566 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.761504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-model-cache\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.761772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.761623    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tmp-dir\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.761772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.761673    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-home\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.761772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.761705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5wq\" (UniqueName: \"kubernetes.io/projected/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-kube-api-access-bh5wq\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.761772 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.761731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tls-certs\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.761944 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.761825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-dshm\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.796684 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.796646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:54:23.862962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.862927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-home\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.862962 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.862970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5wq\" (UniqueName: \"kubernetes.io/projected/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-kube-api-access-bh5wq\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.862989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tls-certs\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.863130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-dshm\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.863212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-model-cache\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863251 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.863253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tmp-dir\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863423 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.863390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-home\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863596 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.863569    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-model-cache\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.863693 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.863630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tmp-dir\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.865561 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.865514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-dshm\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.865694 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.865679    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tls-certs\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.871605 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.871585    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5wq\" (UniqueName: \"kubernetes.io/projected/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-kube-api-access-bh5wq\") pod \"test-model-8ecfce-kserve-866b9d6469-pzwv7\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") " pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:23.963306 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:23.963220    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:24.105644 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:24.105617    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"]
Jul 23 13:54:24.106911 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:54:24.106891    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5f511ae_adee_4db5_ba14_4f1f4fb436c3.slice/crio-eca0eae2fd499210040bac745cf9a92bff81270c623d4cc121f7960f9b3dd0af WatchSource:0}: Error finding container eca0eae2fd499210040bac745cf9a92bff81270c623d4cc121f7960f9b3dd0af: Status 404 returned error can't find the container with id eca0eae2fd499210040bac745cf9a92bff81270c623d4cc121f7960f9b3dd0af
Jul 23 13:54:24.545262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:24.545230    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7" event={"ID":"d5f511ae-adee-4db5-ba14-4f1f4fb436c3","Type":"ContainerStarted","Data":"caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a"}
Jul 23 13:54:24.545262 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:24.545270    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7" event={"ID":"d5f511ae-adee-4db5-ba14-4f1f4fb436c3","Type":"ContainerStarted","Data":"eca0eae2fd499210040bac745cf9a92bff81270c623d4cc121f7960f9b3dd0af"}
Jul 23 13:54:24.563886 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:24.563836    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7" podStartSLOduration=1.563820653 podStartE2EDuration="1.563820653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:54:24.562592252 +0000 UTC m=+2194.938543031" watchObservedRunningTime="2026-07-23 13:54:24.563820653 +0000 UTC m=+2194.939771473"
Jul 23 13:54:33.963891 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:33.963854    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:33.964442 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:33.964166    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:33.976563 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:33.976518    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:54:34.603172 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:54:34.603145    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:55:06.348078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:06.348046    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"]
Jul 23 13:55:06.370944 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:06.370920    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:07.265019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:07.264984    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"]
Jul 23 13:55:07.284198 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:07.284162    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:07.946357 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:07.946322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:08.046938 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:08.046896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:09.127628 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:09.127595    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:09.229203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:09.229158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:18.784294 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:18.784215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:21.369354 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.369303    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7" podUID="09f61fb2-9be1-4f12-a25e-6fcb4b118244" containerName="main" containerID="cri-o://2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154" gracePeriod=45
Jul 23 13:55:21.628841 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.628779    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:55:21.720134 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720090    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-home\") pod \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") "
Jul 23 13:55:21.720134 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720142    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tmp-dir\") pod \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") "
Jul 23 13:55:21.720348 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720182    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7458x\" (UniqueName: \"kubernetes.io/projected/09f61fb2-9be1-4f12-a25e-6fcb4b118244-kube-api-access-7458x\") pod \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") "
Jul 23 13:55:21.720348 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720203    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tls-certs\") pod \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") "
Jul 23 13:55:21.720348 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720249    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-model-cache\") pod \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") "
Jul 23 13:55:21.720489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720361    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-dshm\") pod \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\" (UID: \"09f61fb2-9be1-4f12-a25e-6fcb4b118244\") "
Jul 23 13:55:21.720489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720416    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-home" (OuterVolumeSpecName: "home") pod "09f61fb2-9be1-4f12-a25e-6fcb4b118244" (UID: "09f61fb2-9be1-4f12-a25e-6fcb4b118244"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:21.720489 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720473    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09f61fb2-9be1-4f12-a25e-6fcb4b118244" (UID: "09f61fb2-9be1-4f12-a25e-6fcb4b118244"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:21.720686 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720574    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-model-cache" (OuterVolumeSpecName: "model-cache") pod "09f61fb2-9be1-4f12-a25e-6fcb4b118244" (UID: "09f61fb2-9be1-4f12-a25e-6fcb4b118244"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:21.720752 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720716    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:21.720752 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720732    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:21.720752 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.720745    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:21.722709 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.722671    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-dshm" (OuterVolumeSpecName: "dshm") pod "09f61fb2-9be1-4f12-a25e-6fcb4b118244" (UID: "09f61fb2-9be1-4f12-a25e-6fcb4b118244"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:21.722709 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.722689    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09f61fb2-9be1-4f12-a25e-6fcb4b118244" (UID: "09f61fb2-9be1-4f12-a25e-6fcb4b118244"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:55:21.722709 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.722701    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f61fb2-9be1-4f12-a25e-6fcb4b118244-kube-api-access-7458x" (OuterVolumeSpecName: "kube-api-access-7458x") pod "09f61fb2-9be1-4f12-a25e-6fcb4b118244" (UID: "09f61fb2-9be1-4f12-a25e-6fcb4b118244"). InnerVolumeSpecName "kube-api-access-7458x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:55:21.821236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.821203    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f61fb2-9be1-4f12-a25e-6fcb4b118244-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:21.821236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.821233    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7458x\" (UniqueName: \"kubernetes.io/projected/09f61fb2-9be1-4f12-a25e-6fcb4b118244-kube-api-access-7458x\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:21.821236 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.821242    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f61fb2-9be1-4f12-a25e-6fcb4b118244-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:21.825326 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.825300    2559 generic.go:356] "Generic (PLEG): container finished" podID="09f61fb2-9be1-4f12-a25e-6fcb4b118244" containerID="2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154" exitCode=0
Jul 23 13:55:21.825466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.825347    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"
Jul 23 13:55:21.825466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.825393    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7" event={"ID":"09f61fb2-9be1-4f12-a25e-6fcb4b118244","Type":"ContainerDied","Data":"2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154"}
Jul 23 13:55:21.825466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.825443    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7" event={"ID":"09f61fb2-9be1-4f12-a25e-6fcb4b118244","Type":"ContainerDied","Data":"871bbcef9543745589f8a1cba2e4017be0cb1c8c7c839ca330ab3464e661338c"}
Jul 23 13:55:21.825466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.825465    2559 scope.go:117] "RemoveContainer" containerID="2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154"
Jul 23 13:55:21.835380 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.835357    2559 scope.go:117] "RemoveContainer" containerID="2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154"
Jul 23 13:55:21.835692 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:55:21.835671    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154\": container with ID starting with 2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154 not found: ID does not exist" containerID="2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154"
Jul 23 13:55:21.835754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.835699    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154"} err="failed to get container status \"2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154\": rpc error: code = NotFound desc = could not find container \"2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154\": container with ID starting with 2a87150f6a7b627678df03681dc916a2128a9cf80646e3d3f8f46a3ee7f50154 not found: ID does not exist"
Jul 23 13:55:21.848265 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.848241    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"]
Jul 23 13:55:21.854908 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:21.854887    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-918c39/test-model-918c39-kserve-6b848d58d7-cvcb7"]
Jul 23 13:55:22.092063 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.092033    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09f61fb2-9be1-4f12-a25e-6fcb4b118244" path="/var/lib/kubelet/pods/09f61fb2-9be1-4f12-a25e-6fcb4b118244/volumes"
Jul 23 13:55:22.288272 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.288228    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7" podUID="d5f511ae-adee-4db5-ba14-4f1f4fb436c3" containerName="main" containerID="cri-o://caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a" gracePeriod=45
Jul 23 13:55:22.485416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.485387    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:22.548301 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.548279    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.730860    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tmp-dir\") pod \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") "
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.730921    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-model-cache\") pod \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") "
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.730952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-home\") pod \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") "
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.731011    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-dshm\") pod \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") "
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.731128    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh5wq\" (UniqueName: \"kubernetes.io/projected/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-kube-api-access-bh5wq\") pod \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") "
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.731168    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tls-certs\") pod \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\" (UID: \"d5f511ae-adee-4db5-ba14-4f1f4fb436c3\") "
Jul 23 13:55:22.732203 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.731937    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5f511ae-adee-4db5-ba14-4f1f4fb436c3" (UID: "d5f511ae-adee-4db5-ba14-4f1f4fb436c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:22.733493 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.732629    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5f511ae-adee-4db5-ba14-4f1f4fb436c3" (UID: "d5f511ae-adee-4db5-ba14-4f1f4fb436c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:22.733493 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.732867    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-home" (OuterVolumeSpecName: "home") pod "d5f511ae-adee-4db5-ba14-4f1f4fb436c3" (UID: "d5f511ae-adee-4db5-ba14-4f1f4fb436c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:22.735474 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.735442    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5f511ae-adee-4db5-ba14-4f1f4fb436c3" (UID: "d5f511ae-adee-4db5-ba14-4f1f4fb436c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 13:55:22.738283 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.737726    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-dshm" (OuterVolumeSpecName: "dshm") pod "d5f511ae-adee-4db5-ba14-4f1f4fb436c3" (UID: "d5f511ae-adee-4db5-ba14-4f1f4fb436c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:22.738398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.738372    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-kube-api-access-bh5wq" (OuterVolumeSpecName: "kube-api-access-bh5wq") pod "d5f511ae-adee-4db5-ba14-4f1f4fb436c3" (UID: "d5f511ae-adee-4db5-ba14-4f1f4fb436c3"). InnerVolumeSpecName "kube-api-access-bh5wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:55:22.831830 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.831793    2559 generic.go:356] "Generic (PLEG): container finished" podID="d5f511ae-adee-4db5-ba14-4f1f4fb436c3" containerID="caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a" exitCode=0
Jul 23 13:55:22.831999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.831854    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7" event={"ID":"d5f511ae-adee-4db5-ba14-4f1f4fb436c3","Type":"ContainerDied","Data":"caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a"}
Jul 23 13:55:22.831999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.831868    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"
Jul 23 13:55:22.831999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.831898    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7" event={"ID":"d5f511ae-adee-4db5-ba14-4f1f4fb436c3","Type":"ContainerDied","Data":"eca0eae2fd499210040bac745cf9a92bff81270c623d4cc121f7960f9b3dd0af"}
Jul 23 13:55:22.831999 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.831919    2559 scope.go:117] "RemoveContainer" containerID="caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a"
Jul 23 13:55:22.832231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.832088    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh5wq\" (UniqueName: \"kubernetes.io/projected/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-kube-api-access-bh5wq\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:22.832231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.832142    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:22.832231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.832157    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:22.832231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.832166    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:22.832231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.832184    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:22.832231 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.832196    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f511ae-adee-4db5-ba14-4f1f4fb436c3-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:22.841607 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.841590    2559 scope.go:117] "RemoveContainer" containerID="caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a"
Jul 23 13:55:22.841854 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:55:22.841837    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a\": container with ID starting with caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a not found: ID does not exist" containerID="caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a"
Jul 23 13:55:22.841894 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.841864    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a"} err="failed to get container status \"caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a\": rpc error: code = NotFound desc = could not find container \"caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a\": container with ID starting with caf39b5b010f0628b14d331e9ed12504b05710216d74c314c13f0722ad46cf1a not found: ID does not exist"
Jul 23 13:55:22.853909 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.853875    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"]
Jul 23 13:55:22.859546 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.859516    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8ecfce/test-model-8ecfce-kserve-866b9d6469-pzwv7"]
Jul 23 13:55:22.984012 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:22.983915    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:24.092692 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:24.092663    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f511ae-adee-4db5-ba14-4f1f4fb436c3" path="/var/lib/kubelet/pods/d5f511ae-adee-4db5-ba14-4f1f4fb436c3/volumes"
Jul 23 13:55:28.971146 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:28.971097    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:29.233661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:29.233575    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"]
Jul 23 13:55:29.233804 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:29.233780    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy" containerID="cri-o://633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e" gracePeriod=30
Jul 23 13:55:29.387852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:29.387817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:29.508791 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:29.508705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:29.623091 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:29.623056    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:31.039647 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:31.039614    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:31.133244 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:31.133213    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:34.490466 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.490443    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:55:34.535731 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535661    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081b254f-d274-4430-a3e9-69fc2af179b9-istiod-ca-cert\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.535731 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535713    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081b254f-d274-4430-a3e9-69fc2af179b9-istio-podinfo\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535740    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqrgk\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-kube-api-access-sqrgk\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535788    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-envoy\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535813    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-credential-socket\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535859    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-data\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536016 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.535987    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-istio-token\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536029    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-socket\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536076    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-certs\") pod \"081b254f-d274-4430-a3e9-69fc2af179b9\" (UID: \"081b254f-d274-4430-a3e9-69fc2af179b9\") "
Jul 23 13:55:34.536281 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536170    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:34.536481 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536412    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:34.536565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536475    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.536652 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536629    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:34.537330 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.536075    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081b254f-d274-4430-a3e9-69fc2af179b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:55:34.537414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.537325    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:34.538219 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.538197    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:34.538550 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.538504    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/081b254f-d274-4430-a3e9-69fc2af179b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:55:34.538643 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.538631    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-kube-api-access-sqrgk" (OuterVolumeSpecName: "kube-api-access-sqrgk") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "kube-api-access-sqrgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:55:34.538691 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.538658    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "081b254f-d274-4430-a3e9-69fc2af179b9" (UID: "081b254f-d274-4430-a3e9-69fc2af179b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:55:34.637195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637156    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637188    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637199    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637195 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637207    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081b254f-d274-4430-a3e9-69fc2af179b9-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637215    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081b254f-d274-4430-a3e9-69fc2af179b9-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637222    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqrgk\" (UniqueName: \"kubernetes.io/projected/081b254f-d274-4430-a3e9-69fc2af179b9-kube-api-access-sqrgk\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637232    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.637463 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.637240    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081b254f-d274-4430-a3e9-69fc2af179b9-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:34.898130 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.898096    2559 generic.go:356] "Generic (PLEG): container finished" podID="081b254f-d274-4430-a3e9-69fc2af179b9" containerID="633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e" exitCode=0
Jul 23 13:55:34.898311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.898147    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"
Jul 23 13:55:34.898311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.898146    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" event={"ID":"081b254f-d274-4430-a3e9-69fc2af179b9","Type":"ContainerDied","Data":"633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e"}
Jul 23 13:55:34.898311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.898276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b" event={"ID":"081b254f-d274-4430-a3e9-69fc2af179b9","Type":"ContainerDied","Data":"2e69769f1da363581ffdad435e86432970b7ae289b332307b03121e7c4d887ff"}
Jul 23 13:55:34.898311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.898306    2559 scope.go:117] "RemoveContainer" containerID="633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e"
Jul 23 13:55:34.907778 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.907760    2559 scope.go:117] "RemoveContainer" containerID="372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"
Jul 23 13:55:34.915437 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.915420    2559 scope.go:117] "RemoveContainer" containerID="633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e"
Jul 23 13:55:34.915730 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:55:34.915711    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e\": container with ID starting with 633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e not found: ID does not exist" containerID="633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e"
Jul 23 13:55:34.915784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.915738    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e"} err="failed to get container status \"633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e\": rpc error: code = NotFound desc = could not find container \"633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e\": container with ID starting with 633d3e5858f835e3e00da9bdd52b740c79e189669e2305d89d26af7e5de6924e not found: ID does not exist"
Jul 23 13:55:34.915784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.915755    2559 scope.go:117] "RemoveContainer" containerID="372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"
Jul 23 13:55:34.915986 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:55:34.915971    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194\": container with ID starting with 372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194 not found: ID does not exist" containerID="372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"
Jul 23 13:55:34.916033 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.915990    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194"} err="failed to get container status \"372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194\": rpc error: code = NotFound desc = could not find container \"372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194\": container with ID starting with 372d65177fd8ea6b597ebe5feb9c3c163f663bf1776a30d598f54a3f40bbf194 not found: ID does not exist"
Jul 23 13:55:34.920252 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.920231    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"]
Jul 23 13:55:34.923760 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:34.923740    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-918c39-openshift-default-cf9c46575-v6g4b"]
Jul 23 13:55:36.092178 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:36.092120    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" path="/var/lib/kubelet/pods/081b254f-d274-4430-a3e9-69fc2af179b9/volumes"
Jul 23 13:55:40.340378 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:40.340331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:44.727477 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:44.727441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:45.126048 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:45.126016    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:51.028221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.028174    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:51.126193 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.126157    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:51.286669 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.286585    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"]
Jul 23 13:55:51.286928 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.286880    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" containerID="cri-o://c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f" gracePeriod=30
Jul 23 13:55:51.310956 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.310922    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:51.562106 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.562020    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:51.931334 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:51.931302    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:52.314795 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.314710    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"]
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315074    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f61fb2-9be1-4f12-a25e-6fcb4b118244" containerName="main"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315085    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f61fb2-9be1-4f12-a25e-6fcb4b118244" containerName="main"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315094    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315099    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315110    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315115    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315122    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f511ae-adee-4db5-ba14-4f1f4fb436c3" containerName="main"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f511ae-adee-4db5-ba14-4f1f4fb436c3" containerName="main"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315194    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f511ae-adee-4db5-ba14-4f1f4fb436c3" containerName="main"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315203    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy"
Jul 23 13:55:52.315225 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.315209    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f61fb2-9be1-4f12-a25e-6fcb4b118244" containerName="main"
Jul 23 13:55:52.318022 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.318004    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.321984 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.321961    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-53facf-openshift-default-dockercfg-tk62m\""
Jul 23 13:55:52.369164 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.369127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:52.405678 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-credential-socket\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.405843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istiod-ca-cert\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.405843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-podinfo\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.405843 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405811    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-envoy\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.406040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405866    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-certs\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.406040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-data\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.406040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-socket\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.406040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.405983    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dqrd\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-kube-api-access-9dqrd\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.406040 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.406017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-token\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.413282 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.413252    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"]
Jul 23 13:55:52.506911 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.506879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-credential-socket\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.506927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istiod-ca-cert\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.506947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-podinfo\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.506968    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-envoy\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.506998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-certs\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-data\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-socket\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507117 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dqrd\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-kube-api-access-9dqrd\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507496 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-token\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507496 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507373    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-credential-socket\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507496 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507439    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-certs\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-data\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507717 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-socket\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.507754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.507729    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istiod-ca-cert\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.509863 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.509832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-envoy\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.509989 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.509947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-podinfo\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.516514 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.516489    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-token\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.516904 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.516883    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dqrd\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-kube-api-access-9dqrd\") pod \"e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") " pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.609400 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.609364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:52.628906 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.628880    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:52.714112 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.714071    2559 patch_prober.go:28] interesting pod/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 13:55:52.714283 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.714138    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 13:55:52.772015 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.771990    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"]
Jul 23 13:55:52.773887 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:55:52.773857    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc417a0_c499_41dc_889a_f9ab5b2a3be6.slice/crio-407f1af8628af43214e0137f645c1b050ea26804940b9548e4c0b3ec589d2614 WatchSource:0}: Error finding container 407f1af8628af43214e0137f645c1b050ea26804940b9548e4c0b3ec589d2614: Status 404 returned error can't find the container with id 407f1af8628af43214e0137f645c1b050ea26804940b9548e4c0b3ec589d2614
Jul 23 13:55:52.776416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.776385    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:55:52.776503 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.776451    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:55:52.776503 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.776480    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:55:52.988789 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.988703    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" event={"ID":"7cc417a0-c499-41dc-889a-f9ab5b2a3be6","Type":"ContainerStarted","Data":"a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"}
Jul 23 13:55:52.988789 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:52.988753    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" event={"ID":"7cc417a0-c499-41dc-889a-f9ab5b2a3be6","Type":"ContainerStarted","Data":"407f1af8628af43214e0137f645c1b050ea26804940b9548e4c0b3ec589d2614"}
Jul 23 13:55:53.007801 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:53.007751    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podStartSLOduration=1.00773409 podStartE2EDuration="1.00773409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:55:53.007263415 +0000 UTC m=+2283.383214193" watchObservedRunningTime="2026-07-23 13:55:53.00773409 +0000 UTC m=+2283.383684867"
Jul 23 13:55:53.629771 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:53.629737    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:53.634380 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:53.634357    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:53.993467 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:53.993389    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:53.994288 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:53.994269    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:55:55.100802 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:55.100757    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:55.227124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:55.227089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:55.312721 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:55.312684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:55.414327 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:55.414226    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:55.765433 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:55.765327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:55.865133 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:55.865101    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:55:56.541919 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.541893    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:55:56.648942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.648913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9hqk\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-kube-api-access-h9hqk\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.648942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.648948    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-data\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.648970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-podinfo\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649026    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70de1af2-153e-40e9-8b22-5bb46e94637d-istiod-ca-cert\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649040    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-envoy\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-token\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-certs\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649199 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649175    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-socket\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649507 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649205    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-credential-socket\") pod \"70de1af2-153e-40e9-8b22-5bb46e94637d\" (UID: \"70de1af2-153e-40e9-8b22-5bb46e94637d\") "
Jul 23 13:55:56.649507 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649406    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:56.649634 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649620    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.649686 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649652    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:56.649852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649830    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-data" (OuterVolumeSpecName: "istio-data") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:56.649938 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649912    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70de1af2-153e-40e9-8b22-5bb46e94637d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:55:56.650078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.649991    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:56.651406 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.651377    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:55:56.651406 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.651390    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-kube-api-access-h9hqk" (OuterVolumeSpecName: "kube-api-access-h9hqk") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "kube-api-access-h9hqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:55:56.651567 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.651488    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-token" (OuterVolumeSpecName: "istio-token") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:55:56.651789 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.651769    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70de1af2-153e-40e9-8b22-5bb46e94637d" (UID: "70de1af2-153e-40e9-8b22-5bb46e94637d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:55:56.750983 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.750949    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.750983 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.750978    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.750983 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.750988    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9hqk\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-kube-api-access-h9hqk\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.751206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.750997    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.751206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.751005    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.751206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.751012    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70de1af2-153e-40e9-8b22-5bb46e94637d-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.751206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.751020    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:56.751206 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:56.751028    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70de1af2-153e-40e9-8b22-5bb46e94637d-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:55:57.014985 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.014908    2559 generic.go:356] "Generic (PLEG): container finished" podID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerID="c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f" exitCode=0
Jul 23 13:55:57.014985 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.014957    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"
Jul 23 13:55:57.015174 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.014991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" event={"ID":"70de1af2-153e-40e9-8b22-5bb46e94637d","Type":"ContainerDied","Data":"c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f"}
Jul 23 13:55:57.015174 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.015040    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj" event={"ID":"70de1af2-153e-40e9-8b22-5bb46e94637d","Type":"ContainerDied","Data":"663ccbf5a5604714eea7202c35b05028dfa6fdabaf7af696bb96b760c03f3cd0"}
Jul 23 13:55:57.015174 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.015062    2559 scope.go:117] "RemoveContainer" containerID="c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f"
Jul 23 13:55:57.026207 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.026187    2559 scope.go:117] "RemoveContainer" containerID="d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"
Jul 23 13:55:57.034857 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.034836    2559 scope.go:117] "RemoveContainer" containerID="c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f"
Jul 23 13:55:57.035126 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:55:57.035107    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f\": container with ID starting with c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f not found: ID does not exist" containerID="c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f"
Jul 23 13:55:57.035175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.035135    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f"} err="failed to get container status \"c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f\": rpc error: code = NotFound desc = could not find container \"c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f\": container with ID starting with c87bf3427634d78cd59ea71585ab701c89a3cad15011a038d2dccd67ac3eef1f not found: ID does not exist"
Jul 23 13:55:57.035175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.035155    2559 scope.go:117] "RemoveContainer" containerID="d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"
Jul 23 13:55:57.035382 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:55:57.035367    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a\": container with ID starting with d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a not found: ID does not exist" containerID="d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"
Jul 23 13:55:57.035428 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.035390    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a"} err="failed to get container status \"d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a\": rpc error: code = NotFound desc = could not find container \"d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a\": container with ID starting with d3e6959357cebf29c31c8e389d464c35746239dbfb5a6b01a712af46feb8ce4a not found: ID does not exist"
Jul 23 13:55:57.039058 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.039036    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"]
Jul 23 13:55:57.043361 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:57.043340    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8ecfce-openshift-default-59b8675579-wd5rj"]
Jul 23 13:55:58.091900 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:58.091870    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" path="/var/lib/kubelet/pods/70de1af2-153e-40e9-8b22-5bb46e94637d/volumes"
Jul 23 13:55:59.542597 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:55:59.542565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:00.074936 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.074904    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:00.410622 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.410583    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"]
Jul 23 13:56:00.411438 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.411415    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy"
Jul 23 13:56:00.411564 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.411442    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy"
Jul 23 13:56:00.411683 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.411667    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy"
Jul 23 13:56:00.411739 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.411696    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy"
Jul 23 13:56:00.411739 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.411707    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="081b254f-d274-4430-a3e9-69fc2af179b9" containerName="istio-proxy"
Jul 23 13:56:00.415947 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.415922    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.419675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.419652    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0c8bf4-openshift-default-dockercfg-zmw5d\""
Jul 23 13:56:00.436230 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.436200    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"]
Jul 23 13:56:00.456835 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.456810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:00.485549 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wlwg\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-kube-api-access-2wlwg\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-certs\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-podinfo\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485698 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485682    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-token\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-socket\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485725    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istiod-ca-cert\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-credential-socket\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-data\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.485872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.485847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-envoy\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.586912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.586873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wlwg\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-kube-api-access-2wlwg\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.586912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.586913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-certs\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.586955    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-podinfo\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.586995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-token\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-socket\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istiod-ca-cert\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-credential-socket\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-data\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-envoy\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.587435 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-certs\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.588019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-socket\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.588019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587566    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-credential-socket\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.588019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587614    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-data\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.588019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.587885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istiod-ca-cert\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.589634 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.589605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-envoy\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.589867 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.589843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-podinfo\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.616419 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.616397    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-token\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.617019 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.617002    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wlwg\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-kube-api-access-2wlwg\") pod \"e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") " pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.620654 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.620634    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:00.726653 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.726573    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:00.876875 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.876851    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"]
Jul 23 13:56:00.879527 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:56:00.879500    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfc1347_fcb7_49eb_bde3_ad30a078b9db.slice/crio-70154ec9dceedd2cf73bc711faa5466db4c3c2c1e1ceaa6beac1e6da007b580c WatchSource:0}: Error finding container 70154ec9dceedd2cf73bc711faa5466db4c3c2c1e1ceaa6beac1e6da007b580c: Status 404 returned error can't find the container with id 70154ec9dceedd2cf73bc711faa5466db4c3c2c1e1ceaa6beac1e6da007b580c
Jul 23 13:56:00.881688 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.881657    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:00.881784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.881724    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:00.881784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:00.881758    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:01.037425 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:01.037319    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" event={"ID":"1bfc1347-fcb7-49eb-bde3-ad30a078b9db","Type":"ContainerStarted","Data":"64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"}
Jul 23 13:56:01.037425 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:01.037391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" event={"ID":"1bfc1347-fcb7-49eb-bde3-ad30a078b9db","Type":"ContainerStarted","Data":"70154ec9dceedd2cf73bc711faa5466db4c3c2c1e1ceaa6beac1e6da007b580c"}
Jul 23 13:56:01.064150 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:01.064078    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podStartSLOduration=1.064055568 podStartE2EDuration="1.064055568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:56:01.062609309 +0000 UTC m=+2291.438560091" watchObservedRunningTime="2026-07-23 13:56:01.064055568 +0000 UTC m=+2291.440006347"
Jul 23 13:56:01.727612 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:01.727578    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:01.732458 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:01.732436    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:02.042461 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:02.042376    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:02.043295 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:02.043278    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:03.074477 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:03.074440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:03.176982 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:03.176895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:03.356901 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:03.356869    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:03.553238 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:03.553207    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:03.990622 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:03.990584    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:04.097313 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:04.097272    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:07.627980 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:07.627939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:08.994884 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:08.994847    2559 patch_prober.go:28] interesting pod/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 23 13:56:08.995368 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:08.994909    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 23 13:56:09.078500 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:09.078471    2559 generic.go:356] "Generic (PLEG): container finished" podID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerID="a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a" exitCode=0
Jul 23 13:56:09.078677 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:09.078554    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" event={"ID":"7cc417a0-c499-41dc-889a-f9ab5b2a3be6","Type":"ContainerDied","Data":"a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"}
Jul 23 13:56:09.078943 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:09.078929    2559 scope.go:117] "RemoveContainer" containerID="a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"
Jul 23 13:56:09.079932 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:09.079904    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:09.080030 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:09.079986    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:09.080094 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:09.080035    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:10.104256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:10.104209    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" event={"ID":"7cc417a0-c499-41dc-889a-f9ab5b2a3be6","Type":"ContainerStarted","Data":"a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa"}
Jul 23 13:56:10.630090 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:10.630050    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:56:10.630337 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:10.630311    2559 patch_prober.go:28] interesting pod/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 23 13:56:10.630402 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:10.630382    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 23 13:56:11.629763 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:11.629728    2559 patch_prober.go:28] interesting pod/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 23 13:56:11.630233 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:11.629784    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 23 13:56:12.630229 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:12.630182    2559 patch_prober.go:28] interesting pod/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 23 13:56:12.630661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:12.630256    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 23 13:56:13.630101 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:13.630061    2559 patch_prober.go:28] interesting pod/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 23 13:56:13.630383 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:13.630352    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 23 13:56:14.633124 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:14.633096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:56:14.633622 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:14.633575    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:56:14.634364 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:14.634348    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:56:16.472493 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:16.472455    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:16.580007 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:16.579968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:18.083718 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:18.083672    2559 patch_prober.go:28] interesting pod/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 13:56:18.084159 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:18.083742    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 13:56:19.167132 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:19.167049    2559 generic.go:356] "Generic (PLEG): container finished" podID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerID="64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d" exitCode=0
Jul 23 13:56:19.167523 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:19.167121    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" event={"ID":"1bfc1347-fcb7-49eb-bde3-ad30a078b9db","Type":"ContainerDied","Data":"64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"}
Jul 23 13:56:19.167523 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:19.167516    2559 scope.go:117] "RemoveContainer" containerID="64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"
Jul 23 13:56:19.168593 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:19.168565    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:19.168664 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:19.168638    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:19.168712 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:19.168698    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:56:20.174239 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:20.174203    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" event={"ID":"1bfc1347-fcb7-49eb-bde3-ad30a078b9db","Type":"ContainerStarted","Data":"5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195"}
Jul 23 13:56:20.727087 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:20.727051    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:20.727397 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:20.727371    2559 patch_prober.go:28] interesting pod/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 23 13:56:20.727503 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:20.727429    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 23 13:56:21.727149 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:21.727114    2559 patch_prober.go:28] interesting pod/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 23 13:56:21.727526 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:21.727179    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 23 13:56:22.727777 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:22.727739    2559 patch_prober.go:28] interesting pod/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 23 13:56:22.728181 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:22.727795    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 23 13:56:23.727586 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:23.727527    2559 patch_prober.go:28] interesting pod/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 23 13:56:23.727784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:23.727626    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 23 13:56:24.730792 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:24.730763    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:24.731160 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:24.731019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:24.731710 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:24.731691    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:56:25.826028 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:25.825993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:25.927191 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:25.927153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:31.112046 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:31.112011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:36.000615 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:36.000574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:36.439795 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:36.439765    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:42.523852 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:42.523766    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:42.624913 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:42.624856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:42.787754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:42.787666    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"]
Jul 23 13:56:42.787925 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:42.787873    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" containerID="cri-o://a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa" gracePeriod=30
Jul 23 13:56:42.825950 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:42.825921    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:43.053005 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:43.052921    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:44.635907 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:44.635869    2559 patch_prober.go:28] interesting pod/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 13:56:44.636369 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:44.635950    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 13:56:48.047041 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.047012    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:56:48.153754 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153723    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-socket\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.153942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153779    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-data\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.153942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153818    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-envoy\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.153942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153845    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-credential-socket\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.153942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153862    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dqrd\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-kube-api-access-9dqrd\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.153942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153917    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-podinfo\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.154215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153950    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-token\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.154215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.153991    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istiod-ca-cert\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.154215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154035    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:56:48.154215 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154171    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:56:48.154441 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154334    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-certs\") pod \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\" (UID: \"7cc417a0-c499-41dc-889a-f9ab5b2a3be6\") "
Jul 23 13:56:48.154700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154337    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:56:48.154700 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154582    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:56:48.155036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154718    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.155036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.154736    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.155036 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.155013    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:56:48.156337 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.156312    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:56:48.156462 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.156365    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:56:48.156564 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.156525    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-kube-api-access-9dqrd" (OuterVolumeSpecName: "kube-api-access-9dqrd") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "kube-api-access-9dqrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:56:48.156564 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.156553    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cc417a0-c499-41dc-889a-f9ab5b2a3be6" (UID: "7cc417a0-c499-41dc-889a-f9ab5b2a3be6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:56:48.255923 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255886    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.255923 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255916    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.255923 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255926    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.255923 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255934    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.256196 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255941    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.256196 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255949    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dqrd\" (UniqueName: \"kubernetes.io/projected/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-kube-api-access-9dqrd\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.256196 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.255959    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc417a0-c499-41dc-889a-f9ab5b2a3be6-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:56:48.324742 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.324710    2559 generic.go:356] "Generic (PLEG): container finished" podID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerID="a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa" exitCode=0
Jul 23 13:56:48.324939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.324771    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"
Jul 23 13:56:48.324939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.324787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" event={"ID":"7cc417a0-c499-41dc-889a-f9ab5b2a3be6","Type":"ContainerDied","Data":"a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa"}
Jul 23 13:56:48.324939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.324842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l" event={"ID":"7cc417a0-c499-41dc-889a-f9ab5b2a3be6","Type":"ContainerDied","Data":"407f1af8628af43214e0137f645c1b050ea26804940b9548e4c0b3ec589d2614"}
Jul 23 13:56:48.324939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.324865    2559 scope.go:117] "RemoveContainer" containerID="a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa"
Jul 23 13:56:48.334659 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.334643    2559 scope.go:117] "RemoveContainer" containerID="a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"
Jul 23 13:56:48.342812 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.342793    2559 scope.go:117] "RemoveContainer" containerID="a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa"
Jul 23 13:56:48.343069 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:56:48.343050    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa\": container with ID starting with a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa not found: ID does not exist" containerID="a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa"
Jul 23 13:56:48.343118 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.343078    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa"} err="failed to get container status \"a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa\": rpc error: code = NotFound desc = could not find container \"a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa\": container with ID starting with a94f4b40ea002d26d7ac1e453189b5b1a3d72c29bb214fb5321fee4795e2f6aa not found: ID does not exist"
Jul 23 13:56:48.343118 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.343097    2559 scope.go:117] "RemoveContainer" containerID="a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"
Jul 23 13:56:48.343315 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:56:48.343296    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a\": container with ID starting with a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a not found: ID does not exist" containerID="a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"
Jul 23 13:56:48.343369 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.343325    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a"} err="failed to get container status \"a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a\": rpc error: code = NotFound desc = could not find container \"a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a\": container with ID starting with a5115c96071b5dceb9364647981e1086dee44bd4f61230bb34a3c43bcf349c9a not found: ID does not exist"
Jul 23 13:56:48.349416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.349386    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"]
Jul 23 13:56:48.356590 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:48.356570    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-53facf-openshift-default-54866f7f5-vtl4l"]
Jul 23 13:56:50.091775 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:50.091742    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" path="/var/lib/kubelet/pods/7cc417a0-c499-41dc-889a-f9ab5b2a3be6/volumes"
Jul 23 13:56:51.327023 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:51.326982    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:51.428078 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:51.428033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:53.826628 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:53.826598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:53.921119 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:53.921091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:56.905764 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:56.905732    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:56:59.149501 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:56:59.149464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:03.270017 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:03.269988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:03.676447 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:03.676418    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:10.341171 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:10.341133    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:10.415165 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:10.415120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:10.626768 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:10.626676    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"]
Jul 23 13:57:10.626926 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:10.626900    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy" containerID="cri-o://5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195" gracePeriod=30
Jul 23 13:57:10.655439 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:10.655410    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:10.874412 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:10.874370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:11.241880 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.241846    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:11.613498 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.613465    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"]
Jul 23 13:57:11.614095 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.614075    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy"
Jul 23 13:57:11.614159 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.614097    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="70de1af2-153e-40e9-8b22-5bb46e94637d" containerName="istio-proxy"
Jul 23 13:57:11.614159 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.614109    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy"
Jul 23 13:57:11.614159 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.614115    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy"
Jul 23 13:57:11.614256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.614185    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy"
Jul 23 13:57:11.614256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.614193    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy"
Jul 23 13:57:11.618258 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.618240    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.625165 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.625141    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0cb132-openshift-default-dockercfg-t7hdk\""
Jul 23 13:57:11.643655 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.643632    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"]
Jul 23 13:57:11.659538 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659509    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-podinfo\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659596    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-token\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659661 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659629    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-socket\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659758 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659663    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-certs\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659758 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-credential-socket\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659848 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-data\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659848 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-envoy\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659956 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26pcz\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-kube-api-access-26pcz\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.659956 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.659936    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istiod-ca-cert\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.710900 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.710868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:11.760932 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.760896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-podinfo\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761131 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.760940    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-token\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761131 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.760962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-socket\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761250 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761152    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-certs\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761301 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761257    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-credential-socket\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761301 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761289    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-data\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761409 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761324    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-envoy\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761409 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-socket\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761409 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26pcz\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-kube-api-access-26pcz\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761587 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-certs\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761587 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istiod-ca-cert\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761710 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-credential-socket\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.761887 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.761862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-data\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.762138 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.762114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istiod-ca-cert\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.763552 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.763516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-podinfo\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.763915 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.763899    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-envoy\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.772506 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.772485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26pcz\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-kube-api-access-26pcz\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.772651 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.772632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-token\") pod \"e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") " pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:11.926220 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.926132    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:11.929174 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:11.929156    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:12.071291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.071206    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"]
Jul 23 13:57:12.073648 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:57:12.073621    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35cd91c4_617d_41a2_bd1b_25fcc1f010ff.slice/crio-4df39033ed072fbad9dc2499e5c5ae18e3f995e85392ac3d35261901eaf91b27 WatchSource:0}: Error finding container 4df39033ed072fbad9dc2499e5c5ae18e3f995e85392ac3d35261901eaf91b27: Status 404 returned error can't find the container with id 4df39033ed072fbad9dc2499e5c5ae18e3f995e85392ac3d35261901eaf91b27
Jul 23 13:57:12.075764 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.075734    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:12.075829 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.075804    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:12.075866 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.075831    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:12.445632 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.445590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" event={"ID":"35cd91c4-617d-41a2-bd1b-25fcc1f010ff","Type":"ContainerStarted","Data":"ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"}
Jul 23 13:57:12.445837 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.445652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" event={"ID":"35cd91c4-617d-41a2-bd1b-25fcc1f010ff","Type":"ContainerStarted","Data":"4df39033ed072fbad9dc2499e5c5ae18e3f995e85392ac3d35261901eaf91b27"}
Jul 23 13:57:12.468247 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.468201    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podStartSLOduration=1.468186955 podStartE2EDuration="1.468186955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:57:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:57:12.465802918 +0000 UTC m=+2362.841753694" watchObservedRunningTime="2026-07-23 13:57:12.468186955 +0000 UTC m=+2362.844137733"
Jul 23 13:57:12.929416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:12.929377    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:13.934506 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:13.934476    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:14.429942 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:14.429905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:14.456712 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:14.456682    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:14.457636 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:14.457618    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:14.528558 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:14.528499    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:14.685882 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:14.685779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:14.809495 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:14.809461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:15.147226 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:15.147190    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:15.249906 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:15.249879    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:15.882201 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:15.882177    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:57:16.006020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.005932    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wlwg\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-kube-api-access-2wlwg\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006020 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.005985    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-credential-socket\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006102    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-token\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006141    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-podinfo\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006221 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006200    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-certs\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006232    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-envoy\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006236    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:57:16.006370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006251    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-data\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006288    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-socket\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006370 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006317    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istiod-ca-cert\") pod \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\" (UID: \"1bfc1347-fcb7-49eb-bde3-ad30a078b9db\") "
Jul 23 13:57:16.006681 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006607    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:57:16.006746 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006683    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.006988 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006926    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 13:57:16.006988 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.006962    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:57:16.007186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.007127    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-data" (OuterVolumeSpecName: "istio-data") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:57:16.008492 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.008462    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 13:57:16.008618 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.008465    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 13:57:16.008881 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.008856    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-kube-api-access-2wlwg" (OuterVolumeSpecName: "kube-api-access-2wlwg") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "kube-api-access-2wlwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:57:16.008978 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.008876    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-token" (OuterVolumeSpecName: "istio-token") pod "1bfc1347-fcb7-49eb-bde3-ad30a078b9db" (UID: "1bfc1347-fcb7-49eb-bde3-ad30a078b9db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 13:57:16.107975 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.107946    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.107975 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.107974    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.108168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.107988    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.108168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.107999    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.108168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.108011    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wlwg\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-kube-api-access-2wlwg\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.108168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.108025    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.108168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.108036    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.108168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.108048    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bfc1347-fcb7-49eb-bde3-ad30a078b9db-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 13:57:16.467971 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.467938    2559 generic.go:356] "Generic (PLEG): container finished" podID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerID="5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195" exitCode=0
Jul 23 13:57:16.468416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.467986    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"
Jul 23 13:57:16.468416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.468029    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" event={"ID":"1bfc1347-fcb7-49eb-bde3-ad30a078b9db","Type":"ContainerDied","Data":"5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195"}
Jul 23 13:57:16.468416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.468090    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx" event={"ID":"1bfc1347-fcb7-49eb-bde3-ad30a078b9db","Type":"ContainerDied","Data":"70154ec9dceedd2cf73bc711faa5466db4c3c2c1e1ceaa6beac1e6da007b580c"}
Jul 23 13:57:16.468416 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.468114    2559 scope.go:117] "RemoveContainer" containerID="5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195"
Jul 23 13:57:16.476953 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.476894    2559 scope.go:117] "RemoveContainer" containerID="64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"
Jul 23 13:57:16.484745 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.484730    2559 scope.go:117] "RemoveContainer" containerID="5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195"
Jul 23 13:57:16.485001 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:57:16.484985    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195\": container with ID starting with 5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195 not found: ID does not exist" containerID="5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195"
Jul 23 13:57:16.485051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.485008    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195"} err="failed to get container status \"5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195\": rpc error: code = NotFound desc = could not find container \"5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195\": container with ID starting with 5c12348adfdea5ae22dd999f7fdf7e8150b1288b7589b0d3b914ee82d38f3195 not found: ID does not exist"
Jul 23 13:57:16.485051 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.485024    2559 scope.go:117] "RemoveContainer" containerID="64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"
Jul 23 13:57:16.485244 ip-10-0-136-232 kubenswrapper[2559]: E0723 13:57:16.485226    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d\": container with ID starting with 64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d not found: ID does not exist" containerID="64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"
Jul 23 13:57:16.485284 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.485254    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d"} err="failed to get container status \"64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d\": rpc error: code = NotFound desc = could not find container \"64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d\": container with ID starting with 64ff19aa573766290ff49fc53d868b5f5dcda4e3f4ba727cafd745d1f37f954d not found: ID does not exist"
Jul 23 13:57:16.498515 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.498494    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"]
Jul 23 13:57:16.507093 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.507075    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0c8bf4-openshift-default-77fb5d5c7f-l56dx"]
Jul 23 13:57:16.655393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:16.655362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:18.092103 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:18.092072    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" path="/var/lib/kubelet/pods/1bfc1347-fcb7-49eb-bde3-ad30a078b9db/volumes"
Jul 23 13:57:18.982760 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:18.982723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:19.351100 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.351067    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:19.652084 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.651999    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"]
Jul 23 13:57:19.652650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652618    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy"
Jul 23 13:57:19.652650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652640    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy"
Jul 23 13:57:19.652650 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652652    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy"
Jul 23 13:57:19.652903 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652660    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy"
Jul 23 13:57:19.652903 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652693    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy"
Jul 23 13:57:19.652903 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652702    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc417a0-c499-41dc-889a-f9ab5b2a3be6" containerName="istio-proxy"
Jul 23 13:57:19.652903 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652807    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy"
Jul 23 13:57:19.652903 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.652818    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bfc1347-fcb7-49eb-bde3-ad30a078b9db" containerName="istio-proxy"
Jul 23 13:57:19.657356 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.657335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.660597 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.660574    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3b0b7b-openshift-default-dockercfg-s6glr\""
Jul 23 13:57:19.670380 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.670356    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"]
Jul 23 13:57:19.705027 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.704997    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:19.741270 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741239    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-certs\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-data\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-credential-socket\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741332    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-podinfo\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741414 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741401    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngv2v\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-kube-api-access-ngv2v\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-token\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istiod-ca-cert\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-envoy\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.741641 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.741587    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-socket\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842383 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-credential-socket\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-podinfo\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842565 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngv2v\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-kube-api-access-ngv2v\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842666 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842596    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-token\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842666 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istiod-ca-cert\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842771 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842679    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-envoy\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842771 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-socket\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-certs\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842816    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-data\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.842872 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.842833    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-credential-socket\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.843114 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.843091    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-socket\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.843394 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.843371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istiod-ca-cert\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.843674 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.843425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-certs\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.843743 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.843668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-data\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.845112 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.845093    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-envoy\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.845256 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.845235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-podinfo\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.851191 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.851165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngv2v\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-kube-api-access-ngv2v\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.851349 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.851333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-token\") pod \"e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") " pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:19.880699 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.880665    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:19.967984 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:19.967890    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:20.113797 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.111895    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"]
Jul 23 13:57:20.116517 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:57:20.116489    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a18ada9_0394_4b5e_a0c9_adaec9c00a13.slice/crio-605bb05a148295ee13425012a00776918e4a7695d7d38a617df1788ec3931bfa WatchSource:0}: Error finding container 605bb05a148295ee13425012a00776918e4a7695d7d38a617df1788ec3931bfa: Status 404 returned error can't find the container with id 605bb05a148295ee13425012a00776918e4a7695d7d38a617df1788ec3931bfa
Jul 23 13:57:20.118697 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.118667    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:20.118769 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.118729    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:20.118769 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.118758    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:20.493902 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.493861    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" event={"ID":"0a18ada9-0394-4b5e-a0c9-adaec9c00a13","Type":"ContainerStarted","Data":"3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"}
Jul 23 13:57:20.494428 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.493931    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" event={"ID":"0a18ada9-0394-4b5e-a0c9-adaec9c00a13","Type":"ContainerStarted","Data":"605bb05a148295ee13425012a00776918e4a7695d7d38a617df1788ec3931bfa"}
Jul 23 13:57:20.536444 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.536386    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podStartSLOduration=1.536371138 podStartE2EDuration="1.536371138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:57:20.534074206 +0000 UTC m=+2370.910024986" watchObservedRunningTime="2026-07-23 13:57:20.536371138 +0000 UTC m=+2370.912321917"
Jul 23 13:57:20.968182 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:20.968146    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:21.973905 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:21.973871    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:22.507939 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:22.507905    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:22.510211 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:22.510183    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:24.093140 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:24.093096    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:24.197886 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:24.197837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:27.533748 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:27.533655    2559 generic.go:356] "Generic (PLEG): container finished" podID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerID="ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5" exitCode=0
Jul 23 13:57:27.533748 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:27.533727    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" event={"ID":"35cd91c4-617d-41a2-bd1b-25fcc1f010ff","Type":"ContainerDied","Data":"ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"}
Jul 23 13:57:27.534175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:27.534091    2559 scope.go:117] "RemoveContainer" containerID="ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"
Jul 23 13:57:27.535074 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:27.535050    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:27.535140 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:27.535113    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:27.535186 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:27.535144    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:28.546263 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:28.546230    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" event={"ID":"35cd91c4-617d-41a2-bd1b-25fcc1f010ff","Type":"ContainerStarted","Data":"f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493"}
Jul 23 13:57:28.705569 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:28.705495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:28.929828 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:28.929795    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:28.931112 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:28.931089    2559 patch_prober.go:28] interesting pod/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 23 13:57:28.931175 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:28.931153    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 23 13:57:29.457312 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:29.457279    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:29.929900 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:29.929865    2559 patch_prober.go:28] interesting pod/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 23 13:57:29.930317 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:29.929952    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 23 13:57:30.930396 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:30.930361    2559 patch_prober.go:28] interesting pod/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 23 13:57:30.930793 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:30.930431    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 23 13:57:31.930311 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:31.930264    2559 patch_prober.go:28] interesting pod/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 23 13:57:31.930521 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:31.930335    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 23 13:57:32.933819 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:32.933737    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:32.934841 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:32.934824    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 13:57:35.955633 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:35.955592    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:36.054184 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:36.054143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:38.007784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.007289    2559 patch_prober.go:28] interesting pod/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:39884->10.133.0.62:15021: read: connection reset by peer" start-of-body=
Jul 23 13:57:38.007784 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.007369    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:39884->10.133.0.62:15021: read: connection reset by peer"
Jul 23 13:57:38.604213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.604181    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerID="3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a" exitCode=0
Jul 23 13:57:38.604393 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.604253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" event={"ID":"0a18ada9-0394-4b5e-a0c9-adaec9c00a13","Type":"ContainerDied","Data":"3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"}
Jul 23 13:57:38.604677 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.604664    2559 scope.go:117] "RemoveContainer" containerID="3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"
Jul 23 13:57:38.605694 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.605666    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:38.605796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.605729    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:38.605796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:38.605760    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 13:57:39.611636 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:39.611601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" event={"ID":"0a18ada9-0394-4b5e-a0c9-adaec9c00a13","Type":"ContainerStarted","Data":"9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b"}
Jul 23 13:57:39.968757 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:39.968670    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:39.968924 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:39.968904    2559 patch_prober.go:28] interesting pod/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 23 13:57:39.968977 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:39.968956    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 23 13:57:40.968796 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:40.968753    2559 patch_prober.go:28] interesting pod/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 23 13:57:40.969279 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:40.968831    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 23 13:57:41.968819 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:41.968782    2559 patch_prober.go:28] interesting pod/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 23 13:57:41.969243 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:41.968846    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 23 13:57:42.099834 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.099796    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"]
Jul 23 13:57:42.104363 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.104336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.107681 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.107564    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"openshift-service-ca.crt\""
Jul 23 13:57:42.108816 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.108791    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"auth-test-model-0cb132-kserve-self-signed-certs\""
Jul 23 13:57:42.108981 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.108802    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"default-dockercfg-w55tv\""
Jul 23 13:57:42.109060 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.108914    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"kube-root-ca.crt\""
Jul 23 13:57:42.114189 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.114165    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"]
Jul 23 13:57:42.173749 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.173698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:42.177332 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.177313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-dshm\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.177398 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.177363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-tmp-dir\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.177454 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.177432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-home\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.177513 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.177459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-model-cache\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.177513 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.177500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be4d7154-5520-4718-81c9-ff7f617abcee-tls-certs\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.177651 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.177580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbqnt\" (UniqueName: \"kubernetes.io/projected/be4d7154-5520-4718-81c9-ff7f617abcee-kube-api-access-qbqnt\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-dshm\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279245 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279210    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-tmp-dir\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279451 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-home\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279451 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279271    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-model-cache\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279451 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be4d7154-5520-4718-81c9-ff7f617abcee-tls-certs\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279451 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbqnt\" (UniqueName: \"kubernetes.io/projected/be4d7154-5520-4718-81c9-ff7f617abcee-kube-api-access-qbqnt\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279769 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-home\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.279955 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-tmp-dir\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.280071 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.279800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-model-cache\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.286572 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.282204    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-dshm\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.290188 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.290135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be4d7154-5520-4718-81c9-ff7f617abcee-tls-certs\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.293899 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.293864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbqnt\" (UniqueName: \"kubernetes.io/projected/be4d7154-5520-4718-81c9-ff7f617abcee-kube-api-access-qbqnt\") pod \"auth-test-model-0cb132-kserve-5df9889887-sbzs9\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") " pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.416877 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.416840    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:42.592517 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.592428    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"]
Jul 23 13:57:42.628589 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.628563    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9" event={"ID":"be4d7154-5520-4718-81c9-ff7f617abcee","Type":"ContainerStarted","Data":"dec5dc9c915338d0929be1b3359699f7ffdf9a6f30804d0e66ca6c95e1f3c719"}
Jul 23 13:57:42.969173 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.969081    2559 patch_prober.go:28] interesting pod/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 23 13:57:42.969602 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:42.969176    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 23 13:57:43.634991 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:43.634953    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9" event={"ID":"be4d7154-5520-4718-81c9-ff7f617abcee","Type":"ContainerStarted","Data":"60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060"}
Jul 23 13:57:43.657139 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:43.657090    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9" podStartSLOduration=1.657075443 podStartE2EDuration="1.657075443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:57:43.655281494 +0000 UTC m=+2394.031232273" watchObservedRunningTime="2026-07-23 13:57:43.657075443 +0000 UTC m=+2394.033026220"
Jul 23 13:57:43.972424 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:43.972345    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:43.972839 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:43.972761    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:43.973519 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:43.973502    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 13:57:44.262213 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:44.262115    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:44.363733 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:44.363701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:52.417866 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:52.417831    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:52.417866 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:52.417875    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:52.430009 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:52.429981    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:52.693079 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:52.692994    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 13:57:54.915505 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:54.915455    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:54.974053 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:54.974027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:57:59.753227 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.753186    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"]
Jul 23 13:57:59.756878 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.756856    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.760941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.760905    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"kube-root-ca.crt\""
Jul 23 13:57:59.760941 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.760932    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"default-dockercfg-lsghn\""
Jul 23 13:57:59.761148 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.761076    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"auth-test-model-3b0b7b-kserve-self-signed-certs\""
Jul 23 13:57:59.761202 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.761176    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"openshift-service-ca.crt\""
Jul 23 13:57:59.774291 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.774251    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"]
Jul 23 13:57:59.873626 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.873591    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gnrq\" (UniqueName: \"kubernetes.io/projected/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-kube-api-access-9gnrq\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.873807 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.873640    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-home\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.873807 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.873722    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-dshm\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.873912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.873799    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-model-cache\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.873912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.873848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tls-certs\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.873912 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.873865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tmp-dir\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.974972 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.974934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-dshm\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.974980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-model-cache\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tls-certs\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975017    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tmp-dir\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975091    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnrq\" (UniqueName: \"kubernetes.io/projected/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-kube-api-access-9gnrq\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975168 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-home\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975501 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975473    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-model-cache\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975501 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-home\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.975890 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.975516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tmp-dir\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.977479 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.977458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-dshm\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.977782 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.977763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tls-certs\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:57:59.983762 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:57:59.983734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gnrq\" (UniqueName: \"kubernetes.io/projected/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-kube-api-access-9gnrq\") pod \"auth-test-model-3b0b7b-kserve-5b79994987-b2wwv\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") " pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:58:00.067244 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:00.067155    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:58:00.209494 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:00.209466    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"]
Jul 23 13:58:00.210710 ip-10-0-136-232 kubenswrapper[2559]: W0723 13:58:00.210682    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc02eea7a_2281_4a49_8aa5_1b2ddaf86638.slice/crio-8b35d88f698b99739b43614b5bee1875b09b6ea1a4a59dbadb87d8ffa6aa8164 WatchSource:0}: Error finding container 8b35d88f698b99739b43614b5bee1875b09b6ea1a4a59dbadb87d8ffa6aa8164: Status 404 returned error can't find the container with id 8b35d88f698b99739b43614b5bee1875b09b6ea1a4a59dbadb87d8ffa6aa8164
Jul 23 13:58:00.722483 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:00.722441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv" event={"ID":"c02eea7a-2281-4a49-8aa5-1b2ddaf86638","Type":"ContainerStarted","Data":"1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c"}
Jul 23 13:58:00.722675 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:00.722493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv" event={"ID":"c02eea7a-2281-4a49-8aa5-1b2ddaf86638","Type":"ContainerStarted","Data":"8b35d88f698b99739b43614b5bee1875b09b6ea1a4a59dbadb87d8ffa6aa8164"}
Jul 23 13:58:00.742035 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:00.741992    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv" podStartSLOduration=1.741975203 podStartE2EDuration="1.741975203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:58:00.74073501 +0000 UTC m=+2411.116685788" watchObservedRunningTime="2026-07-23 13:58:00.741975203 +0000 UTC m=+2411.117925982"
Jul 23 13:58:01.583247 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:01.583217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:58:10.068325 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:10.068292    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:58:10.068746 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:10.068337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:58:10.081793 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:10.081768    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:58:10.782067 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:10.782033    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 13:58:13.048606 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:58:13.048469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 13:59:07.080664 ip-10-0-136-232 kubenswrapper[2559]: I0723 13:59:07.080616    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:00:12.123602 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:12.123566    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"]
Jul 23 14:00:12.150791 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:12.150762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:00:12.957706 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:12.957660    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"]
Jul 23 14:00:13.615575 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:13.615522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:00:17.489981 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:17.489930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:00:27.145706 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.145644    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9" podUID="be4d7154-5520-4718-81c9-ff7f617abcee" containerName="main" containerID="cri-o://60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060" gracePeriod=45
Jul 23 14:00:27.404523 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.404467    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 14:00:27.448632 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448599    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be4d7154-5520-4718-81c9-ff7f617abcee-tls-certs\") pod \"be4d7154-5520-4718-81c9-ff7f617abcee\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") "
Jul 23 14:00:27.448812 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448685    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-tmp-dir\") pod \"be4d7154-5520-4718-81c9-ff7f617abcee\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") "
Jul 23 14:00:27.448812 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448728    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-dshm\") pod \"be4d7154-5520-4718-81c9-ff7f617abcee\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") "
Jul 23 14:00:27.448812 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448753    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbqnt\" (UniqueName: \"kubernetes.io/projected/be4d7154-5520-4718-81c9-ff7f617abcee-kube-api-access-qbqnt\") pod \"be4d7154-5520-4718-81c9-ff7f617abcee\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") "
Jul 23 14:00:27.448812 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448781    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-home\") pod \"be4d7154-5520-4718-81c9-ff7f617abcee\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") "
Jul 23 14:00:27.448812 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448796    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-model-cache\") pod \"be4d7154-5520-4718-81c9-ff7f617abcee\" (UID: \"be4d7154-5520-4718-81c9-ff7f617abcee\") "
Jul 23 14:00:27.449149 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.448989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "be4d7154-5520-4718-81c9-ff7f617abcee" (UID: "be4d7154-5520-4718-81c9-ff7f617abcee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:27.449149 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.449098    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-home" (OuterVolumeSpecName: "home") pod "be4d7154-5520-4718-81c9-ff7f617abcee" (UID: "be4d7154-5520-4718-81c9-ff7f617abcee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:27.449268 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.449188    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-model-cache" (OuterVolumeSpecName: "model-cache") pod "be4d7154-5520-4718-81c9-ff7f617abcee" (UID: "be4d7154-5520-4718-81c9-ff7f617abcee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:27.451035 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.451011    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-dshm" (OuterVolumeSpecName: "dshm") pod "be4d7154-5520-4718-81c9-ff7f617abcee" (UID: "be4d7154-5520-4718-81c9-ff7f617abcee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:27.451035 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.451032    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be4d7154-5520-4718-81c9-ff7f617abcee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "be4d7154-5520-4718-81c9-ff7f617abcee" (UID: "be4d7154-5520-4718-81c9-ff7f617abcee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 14:00:27.451193 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.451086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be4d7154-5520-4718-81c9-ff7f617abcee-kube-api-access-qbqnt" (OuterVolumeSpecName: "kube-api-access-qbqnt") pod "be4d7154-5520-4718-81c9-ff7f617abcee" (UID: "be4d7154-5520-4718-81c9-ff7f617abcee"). InnerVolumeSpecName "kube-api-access-qbqnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:00:27.460698 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.460669    2559 generic.go:356] "Generic (PLEG): container finished" podID="be4d7154-5520-4718-81c9-ff7f617abcee" containerID="60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060" exitCode=0
Jul 23 14:00:27.460816 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.460717    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"
Jul 23 14:00:27.460816 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.460763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9" event={"ID":"be4d7154-5520-4718-81c9-ff7f617abcee","Type":"ContainerDied","Data":"60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060"}
Jul 23 14:00:27.460816 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.460807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9" event={"ID":"be4d7154-5520-4718-81c9-ff7f617abcee","Type":"ContainerDied","Data":"dec5dc9c915338d0929be1b3359699f7ffdf9a6f30804d0e66ca6c95e1f3c719"}
Jul 23 14:00:27.460922 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.460825    2559 scope.go:117] "RemoveContainer" containerID="60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060"
Jul 23 14:00:27.472378 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.472357    2559 scope.go:117] "RemoveContainer" containerID="60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060"
Jul 23 14:00:27.472682 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:00:27.472662    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060\": container with ID starting with 60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060 not found: ID does not exist" containerID="60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060"
Jul 23 14:00:27.472763 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.472695    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060"} err="failed to get container status \"60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060\": rpc error: code = NotFound desc = could not find container \"60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060\": container with ID starting with 60ecc559c4fae22753c542d51f4bac901b846d939043dcacd693d78239304060 not found: ID does not exist"
Jul 23 14:00:27.487051 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.487021    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"]
Jul 23 14:00:27.488937 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.488917    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/auth-test-model-0cb132-kserve-5df9889887-sbzs9"]
Jul 23 14:00:27.549505 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.549476    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:27.549505 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.549502    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:27.549686 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.549512    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbqnt\" (UniqueName: \"kubernetes.io/projected/be4d7154-5520-4718-81c9-ff7f617abcee-kube-api-access-qbqnt\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:27.549686 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.549520    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:27.549686 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.549528    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be4d7154-5520-4718-81c9-ff7f617abcee-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:27.549686 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.549553    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be4d7154-5520-4718-81c9-ff7f617abcee-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:27.978695 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:27.978654    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv" podUID="c02eea7a-2281-4a49-8aa5-1b2ddaf86638" containerName="main" containerID="cri-o://1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c" gracePeriod=45
Jul 23 14:00:28.091944 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.091914    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be4d7154-5520-4718-81c9-ff7f617abcee" path="/var/lib/kubelet/pods/be4d7154-5520-4718-81c9-ff7f617abcee/volumes"
Jul 23 14:00:28.246221 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.246196    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 14:00:28.357868 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.357836    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-model-cache\") pod \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") "
Jul 23 14:00:28.358061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.357892    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tmp-dir\") pod \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") "
Jul 23 14:00:28.358061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.357958    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gnrq\" (UniqueName: \"kubernetes.io/projected/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-kube-api-access-9gnrq\") pod \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") "
Jul 23 14:00:28.358061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.357986    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-home\") pod \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") "
Jul 23 14:00:28.358061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358011    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-dshm\") pod \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") "
Jul 23 14:00:28.358277 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358129    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tls-certs\") pod \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\" (UID: \"c02eea7a-2281-4a49-8aa5-1b2ddaf86638\") "
Jul 23 14:00:28.358277 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358144    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-model-cache" (OuterVolumeSpecName: "model-cache") pod "c02eea7a-2281-4a49-8aa5-1b2ddaf86638" (UID: "c02eea7a-2281-4a49-8aa5-1b2ddaf86638"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:28.358277 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358177    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c02eea7a-2281-4a49-8aa5-1b2ddaf86638" (UID: "c02eea7a-2281-4a49-8aa5-1b2ddaf86638"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:28.358413 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358310    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-home" (OuterVolumeSpecName: "home") pod "c02eea7a-2281-4a49-8aa5-1b2ddaf86638" (UID: "c02eea7a-2281-4a49-8aa5-1b2ddaf86638"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:28.358525 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358509    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:28.358590 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358546    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:28.358590 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.358558    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:28.360336 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.360315    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-dshm" (OuterVolumeSpecName: "dshm") pod "c02eea7a-2281-4a49-8aa5-1b2ddaf86638" (UID: "c02eea7a-2281-4a49-8aa5-1b2ddaf86638"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:00:28.360882 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.360863    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c02eea7a-2281-4a49-8aa5-1b2ddaf86638" (UID: "c02eea7a-2281-4a49-8aa5-1b2ddaf86638"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 14:00:28.360964 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.360882    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-kube-api-access-9gnrq" (OuterVolumeSpecName: "kube-api-access-9gnrq") pod "c02eea7a-2281-4a49-8aa5-1b2ddaf86638" (UID: "c02eea7a-2281-4a49-8aa5-1b2ddaf86638"). InnerVolumeSpecName "kube-api-access-9gnrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:00:28.459761 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.459720    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gnrq\" (UniqueName: \"kubernetes.io/projected/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-kube-api-access-9gnrq\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:28.459761 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.459755    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:28.459761 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.459767    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02eea7a-2281-4a49-8aa5-1b2ddaf86638-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:00:28.467745 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.467716    2559 generic.go:356] "Generic (PLEG): container finished" podID="c02eea7a-2281-4a49-8aa5-1b2ddaf86638" containerID="1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c" exitCode=0
Jul 23 14:00:28.467891 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.467752    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"
Jul 23 14:00:28.467891 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.467799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv" event={"ID":"c02eea7a-2281-4a49-8aa5-1b2ddaf86638","Type":"ContainerDied","Data":"1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c"}
Jul 23 14:00:28.467891 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.467835    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv" event={"ID":"c02eea7a-2281-4a49-8aa5-1b2ddaf86638","Type":"ContainerDied","Data":"8b35d88f698b99739b43614b5bee1875b09b6ea1a4a59dbadb87d8ffa6aa8164"}
Jul 23 14:00:28.467891 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.467855    2559 scope.go:117] "RemoveContainer" containerID="1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c"
Jul 23 14:00:28.477550 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.477513    2559 scope.go:117] "RemoveContainer" containerID="1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c"
Jul 23 14:00:28.477824 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:00:28.477805    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c\": container with ID starting with 1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c not found: ID does not exist" containerID="1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c"
Jul 23 14:00:28.477880 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.477832    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c"} err="failed to get container status \"1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c\": rpc error: code = NotFound desc = could not find container \"1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c\": container with ID starting with 1b99430241fcbc96f77070d6d2c2cfe4a32a6430da69cd86b335ec784a3ba75c not found: ID does not exist"
Jul 23 14:00:28.489198 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.489142    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"]
Jul 23 14:00:28.494472 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:28.494449    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/auth-test-model-3b0b7b-kserve-5b79994987-b2wwv"]
Jul 23 14:00:30.091931 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:30.091900    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02eea7a-2281-4a49-8aa5-1b2ddaf86638" path="/var/lib/kubelet/pods/c02eea7a-2281-4a49-8aa5-1b2ddaf86638/volumes"
Jul 23 14:00:50.454043 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454011    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"]
Jul 23 14:00:50.454646 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454592    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4d7154-5520-4718-81c9-ff7f617abcee" containerName="main"
Jul 23 14:00:50.454646 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454612    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4d7154-5520-4718-81c9-ff7f617abcee" containerName="main"
Jul 23 14:00:50.454777 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454654    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c02eea7a-2281-4a49-8aa5-1b2ddaf86638" containerName="main"
Jul 23 14:00:50.454777 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454663    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02eea7a-2281-4a49-8aa5-1b2ddaf86638" containerName="main"
Jul 23 14:00:50.454777 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454750    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4d7154-5520-4718-81c9-ff7f617abcee" containerName="main"
Jul 23 14:00:50.454777 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.454764    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c02eea7a-2281-4a49-8aa5-1b2ddaf86638" containerName="main"
Jul 23 14:00:50.459156 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.459129    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.462331 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.462310    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"openshift-service-ca.crt\""
Jul 23 14:00:50.463243 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.463223    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"sub-test-model-3b0b7b-kserve-self-signed-certs\""
Jul 23 14:00:50.463578 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.463557    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"default-dockercfg-lsghn\""
Jul 23 14:00:50.463647 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.463561    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"kube-root-ca.crt\""
Jul 23 14:00:50.468778 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.468757    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"]
Jul 23 14:00:50.490398 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.490371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:00:50.581372 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.581338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-dshm\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.581610 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.581405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-tmp-dir\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.581610 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.581448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adcecba5-369c-4358-abe1-21ade710efd0-tls-certs\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.581610 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.581509    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ww7b\" (UniqueName: \"kubernetes.io/projected/adcecba5-369c-4358-abe1-21ade710efd0-kube-api-access-9ww7b\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.581762 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.581600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-model-cache\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.581762 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.581646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-home\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682183 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-home\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682384 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-dshm\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682384 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-tmp-dir\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682384 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adcecba5-369c-4358-abe1-21ade710efd0-tls-certs\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682384 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ww7b\" (UniqueName: \"kubernetes.io/projected/adcecba5-369c-4358-abe1-21ade710efd0-kube-api-access-9ww7b\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682637 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-model-cache\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682717 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-home\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682797 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-model-cache\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.682849 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.682833    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-tmp-dir\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.684866 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.684835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-dshm\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.685076 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.685053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adcecba5-369c-4358-abe1-21ade710efd0-tls-certs\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.691108 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.691084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ww7b\" (UniqueName: \"kubernetes.io/projected/adcecba5-369c-4358-abe1-21ade710efd0-kube-api-access-9ww7b\") pod \"sub-test-model-3b0b7b-kserve-67594684b7-m4bht\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") " pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.768772 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.768684    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:00:50.906376 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:50.906344    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"]
Jul 23 14:00:50.909501 ip-10-0-136-232 kubenswrapper[2559]: W0723 14:00:50.909464    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadcecba5_369c_4358_abe1_21ade710efd0.slice/crio-3521bf91e7ff6dacf62035fb867624ec343db755e0827f31c0b60198091c434a WatchSource:0}: Error finding container 3521bf91e7ff6dacf62035fb867624ec343db755e0827f31c0b60198091c434a: Status 404 returned error can't find the container with id 3521bf91e7ff6dacf62035fb867624ec343db755e0827f31c0b60198091c434a
Jul 23 14:00:51.581631 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:51.581590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht" event={"ID":"adcecba5-369c-4358-abe1-21ade710efd0","Type":"ContainerStarted","Data":"68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f"}
Jul 23 14:00:51.581631 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:51.581633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht" event={"ID":"adcecba5-369c-4358-abe1-21ade710efd0","Type":"ContainerStarted","Data":"3521bf91e7ff6dacf62035fb867624ec343db755e0827f31c0b60198091c434a"}
Jul 23 14:00:51.605092 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:00:51.605044    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht" podStartSLOduration=1.605027959 podStartE2EDuration="1.605027959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:00:51.602341557 +0000 UTC m=+2581.978292335" watchObservedRunningTime="2026-07-23 14:00:51.605027959 +0000 UTC m=+2581.980978789"
Jul 23 14:01:00.769118 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:00.769086    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:01:00.769741 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:00.769140    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:01:00.781782 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:00.781755    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:01:01.644922 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:01.644894    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:01:34.038499 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:34.038469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:01:34.830012 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:34.829981    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"]
Jul 23 14:01:34.857255 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:34.857219    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:01:49.850575 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:49.850517    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht" podUID="adcecba5-369c-4358-abe1-21ade710efd0" containerName="main" containerID="cri-o://68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f" gracePeriod=45
Jul 23 14:01:50.107020 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.106954    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:01:50.130405 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130378    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ww7b\" (UniqueName: \"kubernetes.io/projected/adcecba5-369c-4358-abe1-21ade710efd0-kube-api-access-9ww7b\") pod \"adcecba5-369c-4358-abe1-21ade710efd0\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") "
Jul 23 14:01:50.130596 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130432    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adcecba5-369c-4358-abe1-21ade710efd0-tls-certs\") pod \"adcecba5-369c-4358-abe1-21ade710efd0\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") "
Jul 23 14:01:50.130685 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130594    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-tmp-dir\") pod \"adcecba5-369c-4358-abe1-21ade710efd0\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") "
Jul 23 14:01:50.130685 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130628    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-model-cache\") pod \"adcecba5-369c-4358-abe1-21ade710efd0\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") "
Jul 23 14:01:50.130805 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-dshm\") pod \"adcecba5-369c-4358-abe1-21ade710efd0\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") "
Jul 23 14:01:50.130805 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130781    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-home\") pod \"adcecba5-369c-4358-abe1-21ade710efd0\" (UID: \"adcecba5-369c-4358-abe1-21ade710efd0\") "
Jul 23 14:01:50.130918 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130844    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "adcecba5-369c-4358-abe1-21ade710efd0" (UID: "adcecba5-369c-4358-abe1-21ade710efd0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:01:50.130918 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.130860    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-model-cache" (OuterVolumeSpecName: "model-cache") pod "adcecba5-369c-4358-abe1-21ade710efd0" (UID: "adcecba5-369c-4358-abe1-21ade710efd0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:01:50.131344 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.131144    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-home" (OuterVolumeSpecName: "home") pod "adcecba5-369c-4358-abe1-21ade710efd0" (UID: "adcecba5-369c-4358-abe1-21ade710efd0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:01:50.131344 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.131329    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:01:50.131562 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.131351    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:01:50.131562 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.131366    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:01:50.132697 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.132670    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adcecba5-369c-4358-abe1-21ade710efd0-kube-api-access-9ww7b" (OuterVolumeSpecName: "kube-api-access-9ww7b") pod "adcecba5-369c-4358-abe1-21ade710efd0" (UID: "adcecba5-369c-4358-abe1-21ade710efd0"). InnerVolumeSpecName "kube-api-access-9ww7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:01:50.133223 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.133200    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-dshm" (OuterVolumeSpecName: "dshm") pod "adcecba5-369c-4358-abe1-21ade710efd0" (UID: "adcecba5-369c-4358-abe1-21ade710efd0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:01:50.133223 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.133209    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adcecba5-369c-4358-abe1-21ade710efd0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "adcecba5-369c-4358-abe1-21ade710efd0" (UID: "adcecba5-369c-4358-abe1-21ade710efd0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 14:01:50.232358 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.232313    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ww7b\" (UniqueName: \"kubernetes.io/projected/adcecba5-369c-4358-abe1-21ade710efd0-kube-api-access-9ww7b\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:01:50.232358 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.232356    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adcecba5-369c-4358-abe1-21ade710efd0-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:01:50.232358 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.232365    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adcecba5-369c-4358-abe1-21ade710efd0-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:01:50.870372 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.870336    2559 generic.go:356] "Generic (PLEG): container finished" podID="adcecba5-369c-4358-abe1-21ade710efd0" containerID="68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f" exitCode=0
Jul 23 14:01:50.870844 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.870382    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"
Jul 23 14:01:50.870844 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.870430    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht" event={"ID":"adcecba5-369c-4358-abe1-21ade710efd0","Type":"ContainerDied","Data":"68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f"}
Jul 23 14:01:50.870844 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.870477    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht" event={"ID":"adcecba5-369c-4358-abe1-21ade710efd0","Type":"ContainerDied","Data":"3521bf91e7ff6dacf62035fb867624ec343db755e0827f31c0b60198091c434a"}
Jul 23 14:01:50.870844 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.870498    2559 scope.go:117] "RemoveContainer" containerID="68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f"
Jul 23 14:01:50.879841 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.879829    2559 scope.go:117] "RemoveContainer" containerID="68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f"
Jul 23 14:01:50.880105 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:01:50.880085    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f\": container with ID starting with 68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f not found: ID does not exist" containerID="68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f"
Jul 23 14:01:50.880176 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.880120    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f"} err="failed to get container status \"68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f\": rpc error: code = NotFound desc = could not find container \"68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f\": container with ID starting with 68169d20e34c530a9d76e46b160ef6ea1dde8e9e2ef0f316fb8ebd275b76187f not found: ID does not exist"
Jul 23 14:01:50.892341 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.892315    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"]
Jul 23 14:01:50.897116 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:50.897094    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/sub-test-model-3b0b7b-kserve-67594684b7-m4bht"]
Jul 23 14:01:52.094436 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:52.094394    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adcecba5-369c-4358-abe1-21ade710efd0" path="/var/lib/kubelet/pods/adcecba5-369c-4358-abe1-21ade710efd0/volumes"
Jul 23 14:01:52.139857 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:52.139820    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:01:52.240380 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:52.240337    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:01:59.483868 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.483818    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"]
Jul 23 14:01:59.484414 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.484391    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adcecba5-369c-4358-abe1-21ade710efd0" containerName="main"
Jul 23 14:01:59.484414 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.484411    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="adcecba5-369c-4358-abe1-21ade710efd0" containerName="main"
Jul 23 14:01:59.484512 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.484499    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="adcecba5-369c-4358-abe1-21ade710efd0" containerName="main"
Jul 23 14:01:59.487460 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.487441    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.491186 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.491156    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"openshift-service-ca.crt\""
Jul 23 14:01:59.491310 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.491156    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"rate-test-model-0cb132-kserve-self-signed-certs\""
Jul 23 14:01:59.491310 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.491199    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"kube-root-ca.crt\""
Jul 23 14:01:59.491310 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.491158    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0cb132\"/\"default-dockercfg-w55tv\""
Jul 23 14:01:59.498132 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.498110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"]
Jul 23 14:01:59.547514 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.547482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:01:59.618688 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.618649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-model-cache\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.618858 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.618726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-tmp-dir\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.618926 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.618860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-dshm\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.618926 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.618914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48af6685-2d19-480b-9d38-21689b1be414-tls-certs\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.619036 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.618953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsjbk\" (UniqueName: \"kubernetes.io/projected/48af6685-2d19-480b-9d38-21689b1be414-kube-api-access-fsjbk\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.619036 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.619011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-home\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.719966 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.719932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-home\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.719997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-model-cache\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-tmp-dir\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-dshm\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48af6685-2d19-480b-9d38-21689b1be414-tls-certs\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsjbk\" (UniqueName: \"kubernetes.io/projected/48af6685-2d19-480b-9d38-21689b1be414-kube-api-access-fsjbk\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720426 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-home\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720471 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-model-cache\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.720504 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.720466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-tmp-dir\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.722716 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.722693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-dshm\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.722948 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.722929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48af6685-2d19-480b-9d38-21689b1be414-tls-certs\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.728150 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.728128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsjbk\" (UniqueName: \"kubernetes.io/projected/48af6685-2d19-480b-9d38-21689b1be414-kube-api-access-fsjbk\") pod \"rate-test-model-0cb132-kserve-6b4f4b6c9-78xld\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") " pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.797600 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.797495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:01:59.940998 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:01:59.940896    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"]
Jul 23 14:01:59.944015 ip-10-0-136-232 kubenswrapper[2559]: W0723 14:01:59.943988    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48af6685_2d19_480b_9d38_21689b1be414.slice/crio-fd961a62254cbd03ac8c6cdbcfd0c706cc22be2aead382f1e1b8827a69369800 WatchSource:0}: Error finding container fd961a62254cbd03ac8c6cdbcfd0c706cc22be2aead382f1e1b8827a69369800: Status 404 returned error can't find the container with id fd961a62254cbd03ac8c6cdbcfd0c706cc22be2aead382f1e1b8827a69369800
Jul 23 14:02:00.931103 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:00.931065    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld" event={"ID":"48af6685-2d19-480b-9d38-21689b1be414","Type":"ContainerStarted","Data":"31c6eeefc534274fedfd99ed6c1443119a59c7fc47827a0c058399765c6a9146"}
Jul 23 14:02:00.931498 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:00.931116    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld" event={"ID":"48af6685-2d19-480b-9d38-21689b1be414","Type":"ContainerStarted","Data":"fd961a62254cbd03ac8c6cdbcfd0c706cc22be2aead382f1e1b8827a69369800"}
Jul 23 14:02:00.950571 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:00.950506    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld" podStartSLOduration=1.9504906640000002 podStartE2EDuration="1.950490664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:02:00.948966669 +0000 UTC m=+2651.324917447" watchObservedRunningTime="2026-07-23 14:02:00.950490664 +0000 UTC m=+2651.326441643"
Jul 23 14:02:09.797797 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:09.797752    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:02:09.798213 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:09.797816    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:02:09.810595 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:09.810566    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:02:09.985089 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:09.985060    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:02:30.961132 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.961100    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"]
Jul 23 14:02:30.964667 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.964644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:30.968528 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.968497    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"kube-root-ca.crt\""
Jul 23 14:02:30.968528 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.968498    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"openshift-service-ca.crt\""
Jul 23 14:02:30.968748 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.968527    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"default-dockercfg-lsghn\""
Jul 23 14:02:30.968748 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.968496    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3b0b7b\"/\"rate-test-model-3b0b7b-kserve-self-signed-certs\""
Jul 23 14:02:30.976702 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:30.976680    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"]
Jul 23 14:02:31.007291 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.007264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:02:31.023632 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.023604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-model-cache\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.023775 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.023646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df3b1279-25ff-4694-817f-1a4529c8ad8b-tls-certs\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.023775 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.023723    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-tmp-dir\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.023775 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.023765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srzbd\" (UniqueName: \"kubernetes.io/projected/df3b1279-25ff-4694-817f-1a4529c8ad8b-kube-api-access-srzbd\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.023921 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.023794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-dshm\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.023921 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.023869    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-home\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.124656 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.124624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-model-cache\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.124868 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.124771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df3b1279-25ff-4694-817f-1a4529c8ad8b-tls-certs\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.124868 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.124839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-tmp-dir\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.125004 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.124876    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srzbd\" (UniqueName: \"kubernetes.io/projected/df3b1279-25ff-4694-817f-1a4529c8ad8b-kube-api-access-srzbd\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.125004 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.124907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-dshm\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.125004 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.124946    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-home\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.125176 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.125034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-model-cache\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.125230 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.125206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-tmp-dir\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.125298 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.125261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-home\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.127247 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.127220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-dshm\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.127525 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.127505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df3b1279-25ff-4694-817f-1a4529c8ad8b-tls-certs\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.134279 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.134255    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srzbd\" (UniqueName: \"kubernetes.io/projected/df3b1279-25ff-4694-817f-1a4529c8ad8b-kube-api-access-srzbd\") pod \"rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") " pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.275322 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.275249    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:31.425097 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:31.425067    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"]
Jul 23 14:02:31.431185 ip-10-0-136-232 kubenswrapper[2559]: W0723 14:02:31.431159    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf3b1279_25ff_4694_817f_1a4529c8ad8b.slice/crio-e97dbbfbc30e4ba4c0964547c5454eace2ce5ff80e18c6a7ed7068091a51cc07 WatchSource:0}: Error finding container e97dbbfbc30e4ba4c0964547c5454eace2ce5ff80e18c6a7ed7068091a51cc07: Status 404 returned error can't find the container with id e97dbbfbc30e4ba4c0964547c5454eace2ce5ff80e18c6a7ed7068091a51cc07
Jul 23 14:02:32.083959 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:32.083917    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr" event={"ID":"df3b1279-25ff-4694-817f-1a4529c8ad8b","Type":"ContainerStarted","Data":"a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063"}
Jul 23 14:02:32.084384 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:32.083973    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr" event={"ID":"df3b1279-25ff-4694-817f-1a4529c8ad8b","Type":"ContainerStarted","Data":"e97dbbfbc30e4ba4c0964547c5454eace2ce5ff80e18c6a7ed7068091a51cc07"}
Jul 23 14:02:32.102482 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:32.102436    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr" podStartSLOduration=2.102421182 podStartE2EDuration="2.102421182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:02:32.101415727 +0000 UTC m=+2682.477366706" watchObservedRunningTime="2026-07-23 14:02:32.102421182 +0000 UTC m=+2682.478371960"
Jul 23 14:02:36.784741 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:36.784705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:02:36.885311 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:36.885272    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:02:41.276157 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:41.276067    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:41.278645 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:41.276211    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:41.289181 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:41.289157    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:42.142747 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:42.142719    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:02:45.187821 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:45.187774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:02:45.331445 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:02:45.331409    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:17.159776 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:17.159743    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"]
Jul 23 14:03:17.201242 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:17.201213    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:17.971226 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:17.971195    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"]
Jul 23 14:03:17.994242 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:17.994209    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:32.180328 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.180271    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld" podUID="48af6685-2d19-480b-9d38-21689b1be414" containerName="main" containerID="cri-o://31c6eeefc534274fedfd99ed6c1443119a59c7fc47827a0c058399765c6a9146" gracePeriod=45
Jul 23 14:03:32.380641 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.380605    2559 generic.go:356] "Generic (PLEG): container finished" podID="48af6685-2d19-480b-9d38-21689b1be414" containerID="31c6eeefc534274fedfd99ed6c1443119a59c7fc47827a0c058399765c6a9146" exitCode=0
Jul 23 14:03:32.380777 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.380669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld" event={"ID":"48af6685-2d19-480b-9d38-21689b1be414","Type":"ContainerDied","Data":"31c6eeefc534274fedfd99ed6c1443119a59c7fc47827a0c058399765c6a9146"}
Jul 23 14:03:32.438930 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.438878    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:03:32.600366 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600335    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-dshm\") pod \"48af6685-2d19-480b-9d38-21689b1be414\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") "
Jul 23 14:03:32.600551 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600399    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-tmp-dir\") pod \"48af6685-2d19-480b-9d38-21689b1be414\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") "
Jul 23 14:03:32.600551 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600462    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-home\") pod \"48af6685-2d19-480b-9d38-21689b1be414\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") "
Jul 23 14:03:32.600638 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48af6685-2d19-480b-9d38-21689b1be414-tls-certs\") pod \"48af6685-2d19-480b-9d38-21689b1be414\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") "
Jul 23 14:03:32.600638 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsjbk\" (UniqueName: \"kubernetes.io/projected/48af6685-2d19-480b-9d38-21689b1be414-kube-api-access-fsjbk\") pod \"48af6685-2d19-480b-9d38-21689b1be414\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") "
Jul 23 14:03:32.600741 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-model-cache\") pod \"48af6685-2d19-480b-9d38-21689b1be414\" (UID: \"48af6685-2d19-480b-9d38-21689b1be414\") "
Jul 23 14:03:32.600741 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600699    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48af6685-2d19-480b-9d38-21689b1be414" (UID: "48af6685-2d19-480b-9d38-21689b1be414"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:32.600854 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600771    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-home" (OuterVolumeSpecName: "home") pod "48af6685-2d19-480b-9d38-21689b1be414" (UID: "48af6685-2d19-480b-9d38-21689b1be414"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:32.600995 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.600967    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-model-cache" (OuterVolumeSpecName: "model-cache") pod "48af6685-2d19-480b-9d38-21689b1be414" (UID: "48af6685-2d19-480b-9d38-21689b1be414"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:32.601089 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.601031    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:32.601089 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.601046    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:32.602735 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.602711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48af6685-2d19-480b-9d38-21689b1be414-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48af6685-2d19-480b-9d38-21689b1be414" (UID: "48af6685-2d19-480b-9d38-21689b1be414"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 14:03:32.602847 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.602743    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-dshm" (OuterVolumeSpecName: "dshm") pod "48af6685-2d19-480b-9d38-21689b1be414" (UID: "48af6685-2d19-480b-9d38-21689b1be414"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:32.602847 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.602795    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48af6685-2d19-480b-9d38-21689b1be414-kube-api-access-fsjbk" (OuterVolumeSpecName: "kube-api-access-fsjbk") pod "48af6685-2d19-480b-9d38-21689b1be414" (UID: "48af6685-2d19-480b-9d38-21689b1be414"). InnerVolumeSpecName "kube-api-access-fsjbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:03:32.702052 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.701987    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:32.702052 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.702010    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48af6685-2d19-480b-9d38-21689b1be414-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:32.702052 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.702021    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsjbk\" (UniqueName: \"kubernetes.io/projected/48af6685-2d19-480b-9d38-21689b1be414-kube-api-access-fsjbk\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:32.702052 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.702031    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48af6685-2d19-480b-9d38-21689b1be414-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:32.992801 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:32.992680    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr" podUID="df3b1279-25ff-4694-817f-1a4529c8ad8b" containerName="main" containerID="cri-o://a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063" gracePeriod=45
Jul 23 14:03:33.252072 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.252005    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:03:33.386888 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.386854    2559 generic.go:356] "Generic (PLEG): container finished" podID="df3b1279-25ff-4694-817f-1a4529c8ad8b" containerID="a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063" exitCode=0
Jul 23 14:03:33.387057 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.386899    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"
Jul 23 14:03:33.387057 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.386949    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr" event={"ID":"df3b1279-25ff-4694-817f-1a4529c8ad8b","Type":"ContainerDied","Data":"a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063"}
Jul 23 14:03:33.387057 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.387001    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr" event={"ID":"df3b1279-25ff-4694-817f-1a4529c8ad8b","Type":"ContainerDied","Data":"e97dbbfbc30e4ba4c0964547c5454eace2ce5ff80e18c6a7ed7068091a51cc07"}
Jul 23 14:03:33.387057 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.387020    2559 scope.go:117] "RemoveContainer" containerID="a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063"
Jul 23 14:03:33.388284 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.388263    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld" event={"ID":"48af6685-2d19-480b-9d38-21689b1be414","Type":"ContainerDied","Data":"fd961a62254cbd03ac8c6cdbcfd0c706cc22be2aead382f1e1b8827a69369800"}
Jul 23 14:03:33.388355 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.388314    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"
Jul 23 14:03:33.396172 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.396020    2559 scope.go:117] "RemoveContainer" containerID="a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063"
Jul 23 14:03:33.396481 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:03:33.396460    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063\": container with ID starting with a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063 not found: ID does not exist" containerID="a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063"
Jul 23 14:03:33.396571 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.396488    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063"} err="failed to get container status \"a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063\": rpc error: code = NotFound desc = could not find container \"a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063\": container with ID starting with a52702a583e22fcb196271b3218f362f761aef2ef677c90bac9f446ce7924063 not found: ID does not exist"
Jul 23 14:03:33.396571 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.396505    2559 scope.go:117] "RemoveContainer" containerID="31c6eeefc534274fedfd99ed6c1443119a59c7fc47827a0c058399765c6a9146"
Jul 23 14:03:33.410297 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.410271    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"]
Jul 23 14:03:33.412120 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.412090    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0cb132/rate-test-model-0cb132-kserve-6b4f4b6c9-78xld"]
Jul 23 14:03:33.415862 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.415841    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-home\") pod \"df3b1279-25ff-4694-817f-1a4529c8ad8b\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") "
Jul 23 14:03:33.415929 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.415877    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-model-cache\") pod \"df3b1279-25ff-4694-817f-1a4529c8ad8b\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") "
Jul 23 14:03:33.415929 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.415906    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-dshm\") pod \"df3b1279-25ff-4694-817f-1a4529c8ad8b\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") "
Jul 23 14:03:33.416004 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.415951    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srzbd\" (UniqueName: \"kubernetes.io/projected/df3b1279-25ff-4694-817f-1a4529c8ad8b-kube-api-access-srzbd\") pod \"df3b1279-25ff-4694-817f-1a4529c8ad8b\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") "
Jul 23 14:03:33.416100 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416026    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df3b1279-25ff-4694-817f-1a4529c8ad8b-tls-certs\") pod \"df3b1279-25ff-4694-817f-1a4529c8ad8b\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") "
Jul 23 14:03:33.416100 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416045    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-tmp-dir\") pod \"df3b1279-25ff-4694-817f-1a4529c8ad8b\" (UID: \"df3b1279-25ff-4694-817f-1a4529c8ad8b\") "
Jul 23 14:03:33.416210 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416139    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-home" (OuterVolumeSpecName: "home") pod "df3b1279-25ff-4694-817f-1a4529c8ad8b" (UID: "df3b1279-25ff-4694-817f-1a4529c8ad8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:33.416210 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416148    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-model-cache" (OuterVolumeSpecName: "model-cache") pod "df3b1279-25ff-4694-817f-1a4529c8ad8b" (UID: "df3b1279-25ff-4694-817f-1a4529c8ad8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:33.416378 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416356    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-home\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:33.416492 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416386    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-model-cache\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:33.416492 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.416362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df3b1279-25ff-4694-817f-1a4529c8ad8b" (UID: "df3b1279-25ff-4694-817f-1a4529c8ad8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:33.418147 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.418124    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-dshm" (OuterVolumeSpecName: "dshm") pod "df3b1279-25ff-4694-817f-1a4529c8ad8b" (UID: "df3b1279-25ff-4694-817f-1a4529c8ad8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:03:33.418259 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.418156    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df3b1279-25ff-4694-817f-1a4529c8ad8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df3b1279-25ff-4694-817f-1a4529c8ad8b" (UID: "df3b1279-25ff-4694-817f-1a4529c8ad8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 14:03:33.418259 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.418231    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3b1279-25ff-4694-817f-1a4529c8ad8b-kube-api-access-srzbd" (OuterVolumeSpecName: "kube-api-access-srzbd") pod "df3b1279-25ff-4694-817f-1a4529c8ad8b" (UID: "df3b1279-25ff-4694-817f-1a4529c8ad8b"). InnerVolumeSpecName "kube-api-access-srzbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:03:33.517602 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.517503    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-dshm\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:33.517602 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.517555    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srzbd\" (UniqueName: \"kubernetes.io/projected/df3b1279-25ff-4694-817f-1a4529c8ad8b-kube-api-access-srzbd\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:33.517602 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.517571    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df3b1279-25ff-4694-817f-1a4529c8ad8b-tls-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:33.517602 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.517581    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df3b1279-25ff-4694-817f-1a4529c8ad8b-tmp-dir\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:03:33.708560 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.708515    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"]
Jul 23 14:03:33.711615 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:33.711589    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3b0b7b/rate-test-model-3b0b7b-kserve-6fd44c848c-r9jfr"]
Jul 23 14:03:34.091659 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:34.091625    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48af6685-2d19-480b-9d38-21689b1be414" path="/var/lib/kubelet/pods/48af6685-2d19-480b-9d38-21689b1be414/volumes"
Jul 23 14:03:34.092072 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:34.092054    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df3b1279-25ff-4694-817f-1a4529c8ad8b" path="/var/lib/kubelet/pods/df3b1279-25ff-4694-817f-1a4529c8ad8b/volumes"
Jul 23 14:03:37.608089 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:37.608047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:37.680688 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:37.680653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:40.531669 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:40.531632    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:47.595932 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:47.595891    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:52.076233 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:52.076196    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:52.503125 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:52.503096    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:54.709063 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:54.709016    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:58.639703 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:58.639664    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:58.807098 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:58.807065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:58.944850 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:58.944771    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"]
Jul 23 14:03:58.945008 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:58.944985    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy" containerID="cri-o://f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493" gracePeriod=30
Jul 23 14:03:58.966412 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:58.966382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:59.322598 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:59.322473    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:59.847755 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:59.847700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:03:59.949375 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:03:59.949344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:02.936151 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:02.936114    2559 patch_prober.go:28] interesting pod/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 14:04:02.936555 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:02.936188    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 14:04:04.203169 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.203144    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 14:04:04.298792 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298709    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-data\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.298792 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298743    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-podinfo\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.298792 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298764    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-envoy\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299081 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298802    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-certs\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299081 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298834    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-token\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299081 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298856    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-socket\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299081 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298882    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-credential-socket\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299081 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298943    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istiod-ca-cert\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299081 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.298977    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26pcz\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-kube-api-access-26pcz\") pod \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\" (UID: \"35cd91c4-617d-41a2-bd1b-25fcc1f010ff\") "
Jul 23 14:04:04.299388 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299164    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:04.299388 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299180    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:04.299388 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299299    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.299388 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299317    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.299803 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299422    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:04.299803 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299599    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:04.299803 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.299577    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 14:04:04.301200 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.301170    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:04:04.301306 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.301184    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 14:04:04.301306 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.301283    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-kube-api-access-26pcz" (OuterVolumeSpecName: "kube-api-access-26pcz") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "kube-api-access-26pcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:04:04.301398 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.301370    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "35cd91c4-617d-41a2-bd1b-25fcc1f010ff" (UID: "35cd91c4-617d-41a2-bd1b-25fcc1f010ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:04.400407 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400372    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.400407 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400401    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.400407 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400411    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.400661 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400421    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26pcz\" (UniqueName: \"kubernetes.io/projected/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-kube-api-access-26pcz\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.400661 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400431    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.400661 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400438    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.400661 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.400446    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35cd91c4-617d-41a2-bd1b-25fcc1f010ff-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:04.555666 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.555575    2559 generic.go:356] "Generic (PLEG): container finished" podID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerID="f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493" exitCode=0
Jul 23 14:04:04.555666 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.555631    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"
Jul 23 14:04:04.555841 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.555670    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" event={"ID":"35cd91c4-617d-41a2-bd1b-25fcc1f010ff","Type":"ContainerDied","Data":"f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493"}
Jul 23 14:04:04.555841 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.555726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt" event={"ID":"35cd91c4-617d-41a2-bd1b-25fcc1f010ff","Type":"ContainerDied","Data":"4df39033ed072fbad9dc2499e5c5ae18e3f995e85392ac3d35261901eaf91b27"}
Jul 23 14:04:04.555841 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.555749    2559 scope.go:117] "RemoveContainer" containerID="f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493"
Jul 23 14:04:04.565041 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.565024    2559 scope.go:117] "RemoveContainer" containerID="ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"
Jul 23 14:04:04.575464 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.575443    2559 scope.go:117] "RemoveContainer" containerID="f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493"
Jul 23 14:04:04.575739 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:04:04.575716    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493\": container with ID starting with f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493 not found: ID does not exist" containerID="f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493"
Jul 23 14:04:04.575820 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.575747    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493"} err="failed to get container status \"f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493\": rpc error: code = NotFound desc = could not find container \"f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493\": container with ID starting with f90159a0e945419b4b0e65497a2afaf3fbcae1561b47e85717e486a081325493 not found: ID does not exist"
Jul 23 14:04:04.575820 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.575765    2559 scope.go:117] "RemoveContainer" containerID="ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"
Jul 23 14:04:04.576010 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:04:04.575993    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5\": container with ID starting with ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5 not found: ID does not exist" containerID="ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"
Jul 23 14:04:04.576053 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.576016    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5"} err="failed to get container status \"ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5\": rpc error: code = NotFound desc = could not find container \"ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5\": container with ID starting with ca6d194fa8ee195b993f144fe683d62765dbd9bcbfa15b1ffdf932d7c6a94fb5 not found: ID does not exist"
Jul 23 14:04:04.580433 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.580410    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"]
Jul 23 14:04:04.584362 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:04.584342    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0cb132-openshift-default-549fbd4667-2cdbt"]
Jul 23 14:04:06.092323 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:06.092288    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" path="/var/lib/kubelet/pods/35cd91c4-617d-41a2-bd1b-25fcc1f010ff/volumes"
Jul 23 14:04:09.836913 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:09.836871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:14.586551 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:14.586449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:14.798997 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:14.798967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:20.846018 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:20.845983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:20.948879 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:20.948846    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:21.128110 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:21.128018    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"]
Jul 23 14:04:21.128333 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:21.128305    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy" containerID="cri-o://9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b" gracePeriod=30
Jul 23 14:04:21.154576 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:21.154524    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:21.399587 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:21.399490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:26.387684 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.387661    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 14:04:26.500964 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.500892    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-certs\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.500964 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.500941    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-envoy\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.500996    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istiod-ca-cert\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501031    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-socket\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501057    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-podinfo\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501078    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngv2v\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-kube-api-access-ngv2v\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501145 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-token\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501416 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501149    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:26.501416 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501166    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-data\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501416 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501261    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-credential-socket\") pod \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\" (UID: \"0a18ada9-0394-4b5e-a0c9-adaec9c00a13\") "
Jul 23 14:04:26.501605 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501456    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:26.501605 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501476    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 14:04:26.501712 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501666    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-certs\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.501712 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501686    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istiod-ca-cert\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.501712 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501700    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-workload-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.501914 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.501839    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:26.502270 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.502244    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:26.503378 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.503353    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 14:04:26.503493 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.503390    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:04:26.503493 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.503447    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 14:04:26.503761 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.503743    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-kube-api-access-ngv2v" (OuterVolumeSpecName: "kube-api-access-ngv2v") pod "0a18ada9-0394-4b5e-a0c9-adaec9c00a13" (UID: "0a18ada9-0394-4b5e-a0c9-adaec9c00a13"). InnerVolumeSpecName "kube-api-access-ngv2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 14:04:26.602871 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.602843    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-podinfo\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.602871 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.602866    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngv2v\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-kube-api-access-ngv2v\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.602871 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.602877    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-token\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.603061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.602885    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-data\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.603061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.602893    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-credential-socket\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.603061 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.602901    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a18ada9-0394-4b5e-a0c9-adaec9c00a13-istio-envoy\") on node \"ip-10-0-136-232.ec2.internal\" DevicePath \"\""
Jul 23 14:04:26.666993 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.666965    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerID="9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b" exitCode=0
Jul 23 14:04:26.667117 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.667010    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"
Jul 23 14:04:26.667117 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.667041    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" event={"ID":"0a18ada9-0394-4b5e-a0c9-adaec9c00a13","Type":"ContainerDied","Data":"9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b"}
Jul 23 14:04:26.667117 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.667097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh" event={"ID":"0a18ada9-0394-4b5e-a0c9-adaec9c00a13","Type":"ContainerDied","Data":"605bb05a148295ee13425012a00776918e4a7695d7d38a617df1788ec3931bfa"}
Jul 23 14:04:26.667223 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.667120    2559 scope.go:117] "RemoveContainer" containerID="9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b"
Jul 23 14:04:26.676957 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.676942    2559 scope.go:117] "RemoveContainer" containerID="3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"
Jul 23 14:04:26.685068 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.685053    2559 scope.go:117] "RemoveContainer" containerID="9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b"
Jul 23 14:04:26.685325 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:04:26.685307    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b\": container with ID starting with 9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b not found: ID does not exist" containerID="9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b"
Jul 23 14:04:26.685365 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.685333    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b"} err="failed to get container status \"9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b\": rpc error: code = NotFound desc = could not find container \"9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b\": container with ID starting with 9f8b0b749233b9c9df1a65f7ab44382cc1ce416a71de9cfcedab098eb552da3b not found: ID does not exist"
Jul 23 14:04:26.685365 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.685350    2559 scope.go:117] "RemoveContainer" containerID="3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"
Jul 23 14:04:26.685604 ip-10-0-136-232 kubenswrapper[2559]: E0723 14:04:26.685587    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a\": container with ID starting with 3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a not found: ID does not exist" containerID="3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"
Jul 23 14:04:26.685659 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.685611    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a"} err="failed to get container status \"3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a\": rpc error: code = NotFound desc = could not find container \"3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a\": container with ID starting with 3f9685bad325d6baaa25c6b9b77f8f61e2fe564e8401398efc63e70212c2719a not found: ID does not exist"
Jul 23 14:04:26.690358 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.690333    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"]
Jul 23 14:04:26.694298 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:26.694279    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3b0b7b-openshift-default-6559f44b59-2t9rh"]
Jul 23 14:04:27.646451 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:27.646420    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vpdcd_077278b9-ac90-4eb0-9a15-915d41807e89/manager/0.log"
Jul 23 14:04:27.775202 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:27.775171    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-prt2n_17f827da-a6d2-4d6b-9fac-a13e854a6bb3/manager/0.log"
Jul 23 14:04:27.908460 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:27.908387    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fdc9c77b7-l2gkt_1ffc36b1-87cd-4cc0-ab23-16dd30749088/manager/0.log"
Jul 23 14:04:28.054298 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:28.054270    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vb4bl_d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d/server/0.log"
Jul 23 14:04:28.091760 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:28.091729    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" path="/var/lib/kubelet/pods/0a18ada9-0394-4b5e-a0c9-adaec9c00a13/volumes"
Jul 23 14:04:28.188432 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:28.188362    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hqh6w_e0407af5-75a1-42ac-98ca-3d032303f02b/manager/0.log"
Jul 23 14:04:28.597827 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:28.597807    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694f99b88-lrc5l_e0278557-d140-4f4b-92dc-f4dfe56bf6bd/copy-manifests/0.log"
Jul 23 14:04:28.610908 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:28.610889    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694f99b88-lrc5l_e0278557-d140-4f4b-92dc-f4dfe56bf6bd/manager/0.log"
Jul 23 14:04:29.113278 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:29.113248    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746890-xq6sn_b41c1db3-127a-4fcb-8725-e237f29a1e48/cleanup/0.log"
Jul 23 14:04:29.468853 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:29.468779    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fvqtf_29ef5b5b-3e81-4787-885f-6adb17fe362f/postgres/0.log"
Jul 23 14:04:31.008187 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:31.008159    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2hvkb_983c0ead-ef43-4c09-a525-b789cf4ca0f5/manager/0.log"
Jul 23 14:04:31.130211 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:31.130177    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-f54gc_f4c71fd8-40fe-4a40-82be-3ce99a9b4050/manager/0.log"
Jul 23 14:04:31.251680 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:31.251654    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wr9kv_152956f6-7878-40e9-ac9e-405dc84d1eec/kuadrant-console-plugin/0.log"
Jul 23 14:04:31.378593 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:31.378562    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4wh79_9deb41aa-aafe-4dc1-86ef-45eb850eb3f4/registry-server/0.log"
Jul 23 14:04:31.762038 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:31.761964    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rxjs7_e95dccf5-4c7d-4304-871d-ced1afe59c2f/manager/0.log"
Jul 23 14:04:32.141549 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:32.141509    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fzhzk_7c6a1075-948d-4621-b0d8-ce4a65c52cdb/istio-proxy/0.log"
Jul 23 14:04:32.514295 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:32.514204    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75b4c49cfd-sf9xj_e5732263-83cd-4bf2-ba93-98c3d41589e6/kube-auth-proxy/0.log"
Jul 23 14:04:32.644569 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:32.644526    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/1.log"
Jul 23 14:04:32.775033 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:32.774962    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mmvwx_bf34866c-a975-401c-8bdb-2dbd62682f76/payload-pre-processing/0.log"
Jul 23 14:04:32.900654 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:32.900626    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cd4pt_702fe9b2-d746-4577-9caa-120bae5e0518/payload-processing/0.log"
Jul 23 14:04:33.392915 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:33.392884    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5lhql_8f27af21-2fdc-4481-ad08-a980a3de12fd/main/0.log"
Jul 23 14:04:39.956420 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:39.956390    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:40.126473 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:40.126441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log" limit=1024
Jul 23 14:04:42.989362 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:42.989320    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z5rhr_f498563d-5c40-4fdb-b1b1-6b255440eff6/global-pull-secret-syncer/0.log"
Jul 23 14:04:43.041967 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:43.041941    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6tpkf_23df0f51-8206-4026-bdd0-4406375372c9/konnectivity-agent/0.log"
Jul 23 14:04:43.165089 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:43.165053    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-232.ec2.internal_4508d65acded9128b3e84b09c90ba208/haproxy/0.log"
Jul 23 14:04:48.505882 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:48.505850    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2hvkb_983c0ead-ef43-4c09-a525-b789cf4ca0f5/manager/0.log"
Jul 23 14:04:48.533886 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:48.533861    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-f54gc_f4c71fd8-40fe-4a40-82be-3ce99a9b4050/manager/0.log"
Jul 23 14:04:48.562990 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:48.562963    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wr9kv_152956f6-7878-40e9-ac9e-405dc84d1eec/kuadrant-console-plugin/0.log"
Jul 23 14:04:48.611459 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:48.611431    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4wh79_9deb41aa-aafe-4dc1-86ef-45eb850eb3f4/registry-server/0.log"
Jul 23 14:04:48.839043 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:48.839017    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rxjs7_e95dccf5-4c7d-4304-871d-ced1afe59c2f/manager/0.log"
Jul 23 14:04:51.139096 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:51.139072    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f2tj4_dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9/node-exporter/0.log"
Jul 23 14:04:51.182647 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:51.182625    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f2tj4_dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9/kube-rbac-proxy/0.log"
Jul 23 14:04:51.230571 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:51.230528    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f2tj4_dc46b3a1-80b6-4ba4-bea7-47cb0ca305c9/init-textfile/0.log"
Jul 23 14:04:51.950054 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:51.950025    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f6dfcb74b-9jm4s_593c31c9-7381-4b93-8137-b870e191b536/telemeter-client/0.log"
Jul 23 14:04:52.000739 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.000712    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f6dfcb74b-9jm4s_593c31c9-7381-4b93-8137-b870e191b536/reload/0.log"
Jul 23 14:04:52.049889 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.049861    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f6dfcb74b-9jm4s_593c31c9-7381-4b93-8137-b870e191b536/kube-rbac-proxy/0.log"
Jul 23 14:04:52.097734 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.097705    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"]
Jul 23 14:04:52.098072 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098056    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy"
Jul 23 14:04:52.098129 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098076    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy"
Jul 23 14:04:52.098129 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098093    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy"
Jul 23 14:04:52.098129 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098103    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy"
Jul 23 14:04:52.098129 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098118    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy"
Jul 23 14:04:52.098129 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098138    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48af6685-2d19-480b-9d38-21689b1be414" containerName="main"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098146    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="48af6685-2d19-480b-9d38-21689b1be414" containerName="main"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098178    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df3b1279-25ff-4694-817f-1a4529c8ad8b" containerName="main"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098187    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df3b1279-25ff-4694-817f-1a4529c8ad8b" containerName="main"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098286    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df3b1279-25ff-4694-817f-1a4529c8ad8b" containerName="main"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098298    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098306    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098316    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="35cd91c4-617d-41a2-bd1b-25fcc1f010ff" containerName="istio-proxy"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098324    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="48af6685-2d19-480b-9d38-21689b1be414" containerName="main"
Jul 23 14:04:52.098345 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.098332    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a18ada9-0394-4b5e-a0c9-adaec9c00a13" containerName="istio-proxy"
Jul 23 14:04:52.101624 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.101608    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.104221 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.104204    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vk9w9\"/\"kube-root-ca.crt\""
Jul 23 14:04:52.104323 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.104208    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vk9w9\"/\"openshift-service-ca.crt\""
Jul 23 14:04:52.104323 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.104271    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vk9w9\"/\"default-dockercfg-6vdmn\""
Jul 23 14:04:52.111893 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.111870    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"]
Jul 23 14:04:52.226235 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.226143    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-lib-modules\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.226235 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.226177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-proc\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.226712 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.226242    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-podres\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.226712 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.226258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n76d6\" (UniqueName: \"kubernetes.io/projected/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-kube-api-access-n76d6\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.226712 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.226285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-sys\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327590 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-lib-modules\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327590 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327594    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-proc\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327795 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-podres\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327795 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n76d6\" (UniqueName: \"kubernetes.io/projected/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-kube-api-access-n76d6\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327795 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327688    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-sys\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327795 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327751    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-lib-modules\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327795 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327789    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-proc\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327968 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-podres\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.327968 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.327816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-sys\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.335803 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.335778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n76d6\" (UniqueName: \"kubernetes.io/projected/1f99e644-c1e9-4b0f-ab50-254a6fdbecf7-kube-api-access-n76d6\") pod \"perf-node-gather-daemonset-kpk5d\" (UID: \"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7\") " pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.412586 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.412555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.546172 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.546148    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"]
Jul 23 14:04:52.548622 ip-10-0-136-232 kubenswrapper[2559]: W0723 14:04:52.548589    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1f99e644_c1e9_4b0f_ab50_254a6fdbecf7.slice/crio-01d0991b91a46a95c9c6679cd81516675bd1037e44695252c9b3a6a549f6cdae WatchSource:0}: Error finding container 01d0991b91a46a95c9c6679cd81516675bd1037e44695252c9b3a6a549f6cdae: Status 404 returned error can't find the container with id 01d0991b91a46a95c9c6679cd81516675bd1037e44695252c9b3a6a549f6cdae
Jul 23 14:04:52.794230 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.794130    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d" event={"ID":"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7","Type":"ContainerStarted","Data":"fa451bacb0c845e6d9fbdd3cda22f25aaeb0b8c1de70a0e4cffa89926344fc2d"}
Jul 23 14:04:52.794230 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.794181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d" event={"ID":"1f99e644-c1e9-4b0f-ab50-254a6fdbecf7","Type":"ContainerStarted","Data":"01d0991b91a46a95c9c6679cd81516675bd1037e44695252c9b3a6a549f6cdae"}
Jul 23 14:04:52.794230 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.794224    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:52.811901 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:52.811852    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d" podStartSLOduration=0.811834285 podStartE2EDuration="811.834285ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:04:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:04:52.810576715 +0000 UTC m=+2823.186527493" watchObservedRunningTime="2026-07-23 14:04:52.811834285 +0000 UTC m=+2823.187785062"
Jul 23 14:04:54.311926 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:54.311894    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9c8fff85b-nsnh7_3289aa17-967a-43b2-aec8-6d9de1f45656/console/0.log"
Jul 23 14:04:56.151770 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:56.151724    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mv4jn_af2b6355-1e09-42dd-b2b0-df9287eb8be5/dns/0.log"
Jul 23 14:04:56.179465 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:56.179439    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mv4jn_af2b6355-1e09-42dd-b2b0-df9287eb8be5/kube-rbac-proxy/0.log"
Jul 23 14:04:56.212567 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:56.212524    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hvcq5_4c296489-0fc6-4381-87da-d596fad7eaf1/dns-node-resolver/0.log"
Jul 23 14:04:56.925056 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:56.925023    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85bc644f4f-s5h7z_34b9c9e6-7618-4c40-a6a0-ec40f5551d52/registry/0.log"
Jul 23 14:04:56.966627 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:56.966605    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7v5cx_f12762d9-24e8-4482-a8ec-7e294c94cb79/node-ca/0.log"
Jul 23 14:04:57.959665 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:57.959635    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fzhzk_7c6a1075-948d-4621-b0d8-ce4a65c52cdb/istio-proxy/0.log"
Jul 23 14:04:58.150986 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.150948    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75b4c49cfd-sf9xj_e5732263-83cd-4bf2-ba93-98c3d41589e6/kube-auth-proxy/0.log"
Jul 23 14:04:58.179915 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.179883    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/0.log"
Jul 23 14:04:58.208745 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.208718    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/1.log"
Jul 23 14:04:58.237580 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.237482    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mmvwx_bf34866c-a975-401c-8bdb-2dbd62682f76/payload-pre-processing/0.log"
Jul 23 14:04:58.266357 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.266333    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cd4pt_702fe9b2-d746-4577-9caa-120bae5e0518/payload-processing/0.log"
Jul 23 14:04:58.809727 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.809702    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vk9w9/perf-node-gather-daemonset-kpk5d"
Jul 23 14:04:58.981035 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:58.981008    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l5d5t_1ea1d47b-fbc5-49b3-86d3-26397e143e41/serve-healthcheck-canary/0.log"
Jul 23 14:04:59.564118 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:59.564082    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmxhx_acb69f92-4fac-4919-a5b1-023d4cf821f7/kube-rbac-proxy/0.log"
Jul 23 14:04:59.590367 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:59.590341    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmxhx_acb69f92-4fac-4919-a5b1-023d4cf821f7/exporter/0.log"
Jul 23 14:04:59.612305 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:04:59.612286    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmxhx_acb69f92-4fac-4919-a5b1-023d4cf821f7/extractor/0.log"
Jul 23 14:05:02.138273 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.138248    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vpdcd_077278b9-ac90-4eb0-9a15-915d41807e89/manager/0.log"
Jul 23 14:05:02.324439 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.324344    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-prt2n_17f827da-a6d2-4d6b-9fac-a13e854a6bb3/manager/0.log"
Jul 23 14:05:02.443988 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.443957    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fdc9c77b7-l2gkt_1ffc36b1-87cd-4cc0-ab23-16dd30749088/manager/0.log"
Jul 23 14:05:02.465926 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.465901    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vb4bl_d4e95b8e-86e1-4a6d-979c-556e8b5f8f1d/server/0.log"
Jul 23 14:05:02.537208 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.537178    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hqh6w_e0407af5-75a1-42ac-98ca-3d032303f02b/manager/0.log"
Jul 23 14:05:02.670477 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.670437    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fzhzk_7c6a1075-948d-4621-b0d8-ce4a65c52cdb/istio-proxy/0.log"
Jul 23 14:05:02.721762 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.721724    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694f99b88-lrc5l_e0278557-d140-4f4b-92dc-f4dfe56bf6bd/manager/0.log"
Jul 23 14:05:02.759498 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.759470    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694f99b88-lrc5l_e0278557-d140-4f4b-92dc-f4dfe56bf6bd/copy-manifests/0.log"
Jul 23 14:05:02.834834 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.834788    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75b4c49cfd-sf9xj_e5732263-83cd-4bf2-ba93-98c3d41589e6/kube-auth-proxy/0.log"
Jul 23 14:05:02.862086 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.862062    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/0.log"
Jul 23 14:05:02.887466 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.887435    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/1.log"
Jul 23 14:05:02.906796 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.906773    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mmvwx_bf34866c-a975-401c-8bdb-2dbd62682f76/payload-pre-processing/0.log"
Jul 23 14:05:02.926550 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:02.926462    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cd4pt_702fe9b2-d746-4577-9caa-120bae5e0518/payload-processing/0.log"
Jul 23 14:05:04.095108 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:04.095080    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fzhzk_7c6a1075-948d-4621-b0d8-ce4a65c52cdb/istio-proxy/0.log"
Jul 23 14:05:04.256284 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:04.256254    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75b4c49cfd-sf9xj_e5732263-83cd-4bf2-ba93-98c3d41589e6/kube-auth-proxy/0.log"
Jul 23 14:05:04.283849 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:04.283822    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/0.log"
Jul 23 14:05:04.310482 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:04.310453    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bn594_35b6cbcf-290d-4c3b-81ee-92b5c7b6f852/istio-proxy/1.log"
Jul 23 14:05:04.331724 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:04.331689    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mmvwx_bf34866c-a975-401c-8bdb-2dbd62682f76/payload-pre-processing/0.log"
Jul 23 14:05:04.356010 ip-10-0-136-232 kubenswrapper[2559]: I0723 14:05:04.355932    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cd4pt_702fe9b2-d746-4577-9caa-120bae5e0518/payload-processing/0.log"