Jul 18 13:30:26.583475 ip-10-0-128-123 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 18 13:30:26.583487 ip-10-0-128-123 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 18 13:30:26.583496 ip-10-0-128-123 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 18 13:30:26.583835 ip-10-0-128-123 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 18 13:30:36.601355 ip-10-0-128-123 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 18 13:30:36.601370 ip-10-0-128-123 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1d558fde60f443ff95b3ba84a8f1d108 --
Jul 18 13:32:30.237313 ip-10-0-128-123 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 13:32:30.594680 ip-10-0-128-123 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 13:32:30.594680 ip-10-0-128-123 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 13:32:30.594680 ip-10-0-128-123 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 13:32:30.594680 ip-10-0-128-123 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 13:32:30.594680 ip-10-0-128-123 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 13:32:30.596344 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.596267    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 13:32:30.601152 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601138    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:30.601152 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601152    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:30.601218 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601157    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:30.601218 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601160    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:30.601218 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601163    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:30.601739 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601731    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:30.601739 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601739    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601743    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601746    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601749    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601753    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601755    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601758    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601766    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601769    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601773    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601775    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601778    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601781    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601783    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601786    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601788    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601791    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601796    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601800    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:30.601795 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601803    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601807    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601811    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601814    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601817    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601820    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601823    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601826    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601828    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601831    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601834    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601836    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601838    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601841    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601843    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601846    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601849    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601853    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601855    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:30.602253 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601858    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601861    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601864    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601866    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601869    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601871    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601873    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601876    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601878    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601881    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601883    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601886    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601888    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601891    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601894    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601900    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601903    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601906    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601909    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601911    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:30.602711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601914    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601916    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601919    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601922    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601924    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601927    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601929    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601932    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601934    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601938    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601940    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601943    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601945    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601947    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601950    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601953    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601955    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601957    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601960    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601963    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:30.603193 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601966    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601968    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601971    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601973    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601976    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601978    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601981    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601984    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601987    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601989    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601992    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601994    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.601997    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602000    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602002    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602692    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602699    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602703    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602706    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:30.603753 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602708    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602711    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602714    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602717    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602719    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602722    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602726    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602730    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602733    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602736    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602739    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602742    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602745    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602748    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602750    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602753    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602757    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602759    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602762    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602764    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:30.604232 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602768    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602771    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602774    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602776    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602779    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602782    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602785    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602788    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602791    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602794    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602797    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602799    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602802    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602804    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602807    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602809    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602811    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602815    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602817    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:30.604709 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602820    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602822    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602825    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602828    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602831    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602833    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602836    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602839    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602842    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602844    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602847    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602851    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602854    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602858    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602860    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602863    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602866    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602869    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602872    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:30.605192 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602875    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602877    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602880    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602882    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602885    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602887    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602890    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602892    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602895    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602897    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602900    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602902    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602905    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602908    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602911    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602913    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602916    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602919    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602921    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602924    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:30.605653 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602927    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602929    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602932    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602934    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602937    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602939    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602942    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602944    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602947    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602949    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602951    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602954    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602957    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602959    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602962    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602964    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.602967    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603340    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603345    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603348    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:30.606151 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603351    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603354    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603358    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603360    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603363    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603366    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603369    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603372    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603374    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603377    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603380    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603383    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603385    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603388    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603390    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603393    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603395    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603398    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603400    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603403    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:30.606649 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603407    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603410    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603413    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603415    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603418    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603420    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603423    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603425    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603428    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603431    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603433    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603437    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603440    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603443    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603445    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603448    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603450    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603453    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603455    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:30.607141 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603458    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603462    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603465    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603469    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603471    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603474    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603476    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603479    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603481    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603483    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603486    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603488    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603491    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603496    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603498    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603501    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603504    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603506    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603509    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:30.607588 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603511    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603514    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603516    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603518    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603521    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603526    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603529    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603531    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603534    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603536    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603541    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603544    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603547    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603550    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603553    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603555    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603558    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603561    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603563    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603566    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:30.608095 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603568    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603571    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603573    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603577    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603580    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603583    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603587    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603590    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603593    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603596    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603598    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603601    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603603    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603606    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603608    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603611    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603613    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.603617    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603698    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603708    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603717    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 13:32:30.608566 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603724    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603734    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603738    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603742    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603745    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603748    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603751    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603755    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603758    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603761    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603764    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603768    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603771    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603773    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603776    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603780    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603783    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603788    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603791    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603794    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603798    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603801    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603804    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 13:32:30.609096 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603807    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603810    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603813    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603816    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603819    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603824    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603827    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603830    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603833    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603836    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603839    2577 flags.go:64] FLAG: --enable-server="true"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603842    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603846    2577 flags.go:64] FLAG: --event-burst="100"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603849    2577 flags.go:64] FLAG: --event-qps="50"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603852    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603855    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603858    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603862    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603864    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603868    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603871    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603874    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603877    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603880    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603882    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 13:32:30.609666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603885    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603890    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603893    2577 flags.go:64] FLAG: --feature-gates=""
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603896    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603899    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603902    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603905    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603908    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603911    2577 flags.go:64] FLAG: --help="false"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603914    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603917    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603920    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603924    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603927    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603930    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603933    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603936    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603939    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603942    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603944    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603947    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603950    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603953    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603956    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 13:32:30.610401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603959    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603961    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603964    2577 flags.go:64] FLAG: --lock-file=""
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603967    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603970    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603973    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603978    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603981    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603983    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603987    2577 flags.go:64] FLAG: --logging-format="text"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603990    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603993    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603996    2577 flags.go:64] FLAG: --manifest-url=""
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.603999    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604003    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604006    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604010    2577 flags.go:64] FLAG: --max-pods="110"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604013    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604015    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604018    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604022    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604026    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604028    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604031    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604054    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 13:32:30.611363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604059    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604063    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604067    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604072    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604080    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604083    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604087    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604089    2577 flags.go:64] FLAG: --port="10250"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604092    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604095    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03cbe4b63dc0bad96"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604098    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604102    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604105    2577 flags.go:64] FLAG: --register-node="true"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604108    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604111    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604114    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604118    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604121    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604125    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604128    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604131    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604133    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604136    2577 flags.go:64] FLAG: --runonce="false"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604139    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604142    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 13:32:30.611995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604145    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604148    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604151    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604155    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604158    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604161    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604164    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604167    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604170    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604172    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604175    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604178    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604181    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604186    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604189    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604191    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604196    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604199    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604202    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604205    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604208    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604211    2577 flags.go:64] FLAG: --v="2"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604215    2577 flags.go:64] FLAG: --version="false"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604220    2577 flags.go:64] FLAG: --vmodule=""
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604224    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 13:32:30.612613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604227    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604335    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604339    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604342    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604344    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604347    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604349    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604352    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604355    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604357    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604360    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604363    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604366    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604368    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604371    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604373    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604378    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604381    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604384    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604387    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:30.613229 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604390    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604392    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604395    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604397    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604400    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604402    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604405    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604409    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604411    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604414    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604417    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604419    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604422    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604424    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604427    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604430    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604434    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604437    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604439    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:30.613740 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604442    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604444    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604447    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604450    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604452    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604455    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604458    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604460    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604462    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604465    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604468    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604470    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604473    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604475    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604478    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604480    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604483    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604485    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604488    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604490    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:30.614216 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604493    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604496    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604498    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604501    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604504    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604506    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604509    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604512    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604514    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604517    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604519    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604523    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604526    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604529    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604532    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604535    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604538    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604541    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604543    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:30.614702 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604546    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604548    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604551    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604553    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604556    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604558    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604561    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604564    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604566    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604569    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604571    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604574    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604576    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604579    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604581    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604584    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604587    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604589    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604592    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604594    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:30.615168 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604596    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.604599    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.604604    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.612613    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.612628    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.613376    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.613386    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613441    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613446    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613450    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613453    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613457    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613460    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613463    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613465    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:30.615629 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613468    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613470    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613473    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613476    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613479    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613481    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613484    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613486    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613488    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613491    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613494    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613496    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613499    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613506    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613509    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613512    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613514    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613517    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613519    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:30.615991 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613522    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613524    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613529    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613533    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613536    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613538    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613541    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613544    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613546    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613551    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613554    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613558    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613561    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613564    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613568    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613571    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613574    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613577    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613581    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:30.616460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613584    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613587    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613589    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613592    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613594    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613597    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613599    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613601    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613604    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613607    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613609    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613611    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613614    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613616    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613619    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613622    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613625    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613628    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613630    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613633    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:30.616910 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613636    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613639    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613641    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613644    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613646    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613649    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613651    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613654    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613656    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613659    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613661    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613664    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613666    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613669    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613672    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613674    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613677    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613679    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613682    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:30.617484 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613684    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613687    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613689    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613692    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613694    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613697    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613699    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613702    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613705    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613708    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613710    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613713    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613715    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613718    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.613723    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 13:32:30.617937 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613850    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613856    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613859    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613862    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613864    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613867    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613870    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613872    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613875    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613877    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613880    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613882    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613885    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613888    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613890    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613893    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613895    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613898    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613900    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613902    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 13:32:30.618320 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613905    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613907    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613910    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613913    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613932    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613936    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613939    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613942    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613945    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613948    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613951    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613954    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613957    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613960    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613962    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613965    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613968    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613971    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613974    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 13:32:30.618780 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613977    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613979    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613982    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613984    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613987    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613991    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613996    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.613999    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614002    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614005    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614007    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614010    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614013    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614016    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614018    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614021    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614024    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614026    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614029    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 13:32:30.619262 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614032    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614050    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614053    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614057    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614060    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614063    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614066    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614070    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614073    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614076    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614079    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614082    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614084    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614087    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614089    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614092    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614094    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614097    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614099    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 13:32:30.619711 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614102    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614105    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614108    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614111    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614113    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614116    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614118    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614121    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614123    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614126    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614128    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614131    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614133    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614136    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614138    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614141    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614143    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614145    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614148    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614150    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 13:32:30.620275 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614153    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:30.614155    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.614160    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.614257    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.617172    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.618707    2577 server.go:1019] "Starting client certificate rotation"
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.618802    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 13:32:30.620741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.618835    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 13:32:30.621604 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.621590    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 13:32:30.622662 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.622649    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 13:32:30.630751 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.630735    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 18 13:32:30.633441 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.633426    2577 log.go:25] "Validated CRI v1 image API"
Jul 18 13:32:30.634814 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.634796    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 13:32:30.636771 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.636748    2577 fs.go:135] Filesystem UUIDs: map[2b4edb27-c7f6-4507-8376-b22b3fce7267:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a903fefc-f563-4f6c-89f8-b3f2a67c22f5:/dev/nvme0n1p4]
Jul 18 13:32:30.636836 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.636771    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 13:32:30.643826 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.643708    2577 manager.go:216] Machine: {Timestamp:2026-07-18 13:32:30.641933913 +0000 UTC m=+0.298624037 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105815 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25e73b86d868d4d13f605895444b8a SystemUUID:ec25e73b-86d8-68d4-d13f-605895444b8a BootID:1d558fde-60f4-43ff-95b3-ba84a8f1d108 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:42:b9:ad:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:42:b9:ad:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:72:e4:0e:2e:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 13:32:30.643826 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.643819    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 13:32:30.643952 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.643862    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 13:32:30.643952 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.643943    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 13:32:30.646767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.646742    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 13:32:30.646891 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.646768    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 13:32:30.647518 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.647508    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 13:32:30.647554 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.647520    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 13:32:30.647554 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.647526    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 13:32:30.648296 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.648286    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 13:32:30.649995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.649985    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 13:32:30.653318 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653307    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 13:32:30.653418 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653409    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 13:32:30.653451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653438    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 13:32:30.653451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653447    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 13:32:30.653503 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653460    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 13:32:30.653503 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653468    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 18 13:32:30.653503 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653476    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 13:32:30.653816 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653804    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 13:32:30.653861 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.653823    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 13:32:30.655707 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.655693    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 13:32:30.657960 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.657947    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 13:32:30.658015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.657976    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 13:32:30.659169 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659155    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659172    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659178    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659183    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659193    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659198    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659207    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659212    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659218    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659223    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659233    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 13:32:30.659233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.659238    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 13:32:30.660534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.660522    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 13:32:30.660534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.660533    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 13:32:30.660815 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.660787    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 13:32:30.660897 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.660855    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 13:32:30.663554 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.663542    2577 server.go:1268] "Started kubelet"
Jul 18 13:32:30.663636 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.663610    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 13:32:30.663703 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.663609    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 13:32:30.663703 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.663671    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 13:32:30.663703 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.663677    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xtcc2"
Jul 18 13:32:30.664342 ip-10-0-128-123 systemd[1]: Started Kubernetes Kubelet.
Jul 18 13:32:30.664633 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.664614    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 13:32:30.665527 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.665509    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 13:32:30.665851 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.665835    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 13:32:30.668768 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.668742    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xtcc2"
Jul 18 13:32:30.669414 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.669398    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 13:32:30.670111 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.669205    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-123.ec2.internal.18c36590f70f438c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-123.ec2.internal,UID:ip-10-0-128-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-123.ec2.internal,},FirstTimestamp:2026-07-18 13:32:30.663517068 +0000 UTC m=+0.320207187,LastTimestamp:2026-07-18 13:32:30.663517068 +0000 UTC m=+0.320207187,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-123.ec2.internal,}"
Jul 18 13:32:30.674682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.674667    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 13:32:30.674722 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.674682    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 13:32:30.674772 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.674737    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:30.675241 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675192    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 13:32:30.675241 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675209    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 13:32:30.675241 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675224    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 13:32:30.675618 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675349    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 13:32:30.675618 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675356    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 13:32:30.675883 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675865    2577 factory.go:55] Registering systemd factory
Jul 18 13:32:30.675944 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.675886    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 18 13:32:30.676131 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.676112    2577 factory.go:153] Registering CRI-O factory
Jul 18 13:32:30.676131 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.676131    2577 factory.go:223] Registration of the crio container factory successfully
Jul 18 13:32:30.676284 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.676195    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 13:32:30.676284 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.676226    2577 factory.go:103] Registering Raw factory
Jul 18 13:32:30.676284 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.676239    2577 manager.go:1196] Started watching for new ooms in manager
Jul 18 13:32:30.676412 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.676315    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 18 13:32:30.676452 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.676432    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 18 13:32:30.676690 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.676667    2577 manager.go:349] Starting recovery of all containers
Jul 18 13:32:30.687735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.687720    2577 manager.go:354] Recovery completed
Jul 18 13:32:30.692326 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.692315    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.694014 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.693998    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.694130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.694024    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.694130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.694047    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.694596 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.694582    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 13:32:30.694678 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.694599    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.694678 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.694602    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 13:32:30.694678 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.694621    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 13:32:30.695254 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.695236    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.695316 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.695264    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.695316 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.695275    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.695738 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.695722    2577 policy_none.go:49] "None policy: Start"
Jul 18 13:32:30.695795 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.695745    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 13:32:30.695795 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.695761    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 13:32:30.696731 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.696719    2577 policy_none.go:47] "Start"
Jul 18 13:32:30.727651 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.727527    2577 manager.go:337] "Starting Device Plugin manager"
Jul 18 13:32:30.727753 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.727679    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 13:32:30.727753 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.727692    2577 server.go:87] "Starting device plugin registration server"
Jul 18 13:32:30.727924 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.727912    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 13:32:30.727982 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.727926    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 13:32:30.728056 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.728025    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 13:32:30.728137 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.728123    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 13:32:30.728137 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.728136    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 13:32:30.729187 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.729167    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 13:32:30.729267 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.729214    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:30.794253 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.794216    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 13:32:30.795459 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.795444    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 13:32:30.795533 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.795486    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 13:32:30.795533 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.795515    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 13:32:30.795631 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.795553    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 13:32:30.797449 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.797428    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:30.828089 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.828069    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.828910 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.828896    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.828981 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.828924    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.828981 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.828941    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.828981 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.828965    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.835865 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.835846    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.835945 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.835870    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-123.ec2.internal\": node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:30.850285 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.850262    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:30.896424 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.896363    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"]
Jul 18 13:32:30.896424 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.896416    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.897254 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.897234    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.897343 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.897260    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.897343 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.897269    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.898406 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.898394    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.898533 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.898520    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.898577 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.898559    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.899061 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.899029    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.899061 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.899067    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.899203 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.899079    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.899203 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.899090    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.899203 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.899106    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.899203 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.899114    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.900395 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.900379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.900459 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.900410    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 13:32:30.901005 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.900990    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 13:32:30.901095 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.901013    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 13:32:30.901095 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.901024    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID"
Jul 18 13:32:30.931957 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.931934    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-123.ec2.internal\" not found" node="ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.936952 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.936934    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-123.ec2.internal\" not found" node="ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.951143 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:30.951118    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:30.977791 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.977760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8926567829fff2679ba8a397c742012f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-123.ec2.internal\" (UID: \"8926567829fff2679ba8a397c742012f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.977791 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.977789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/da14a40c1d1653f9f035a1d464909031-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"da14a40c1d1653f9f035a1d464909031\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:30.977957 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:30.977807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da14a40c1d1653f9f035a1d464909031-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"da14a40c1d1653f9f035a1d464909031\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.053370 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.053328    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.078398 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.078374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8926567829fff2679ba8a397c742012f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-123.ec2.internal\" (UID: \"8926567829fff2679ba8a397c742012f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.078481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.078402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/da14a40c1d1653f9f035a1d464909031-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"da14a40c1d1653f9f035a1d464909031\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.078481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.078421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da14a40c1d1653f9f035a1d464909031-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"da14a40c1d1653f9f035a1d464909031\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.078481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.078459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da14a40c1d1653f9f035a1d464909031-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"da14a40c1d1653f9f035a1d464909031\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.078586 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.078484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8926567829fff2679ba8a397c742012f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-123.ec2.internal\" (UID: \"8926567829fff2679ba8a397c742012f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.078586 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.078490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/da14a40c1d1653f9f035a1d464909031-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"da14a40c1d1653f9f035a1d464909031\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.155524 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.155433    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.240361 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.240313    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.240361 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.240347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:31.255531 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.255507    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.356919 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.356879    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.457367 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.457286    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.558283 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.558255    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.618317 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.618282    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 13:32:31.618748 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.618437    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 13:32:31.658980 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.658955    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.674118 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.674067    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 13:27:31 +0000 UTC" deadline="2028-01-28 03:51:01.097887667 +0000 UTC"
Jul 18 13:32:31.674118 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.674113    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13406h18m29.423778076s"
Jul 18 13:32:31.759618 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.759553    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.795286 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:31.795257    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda14a40c1d1653f9f035a1d464909031.slice/crio-a640bffe5076b6902b0ad549c3d41e8521c7321f09373341949bde5bc0ee9333 WatchSource:0}: Error finding container a640bffe5076b6902b0ad549c3d41e8521c7321f09373341949bde5bc0ee9333: Status 404 returned error can't find the container with id a640bffe5076b6902b0ad549c3d41e8521c7321f09373341949bde5bc0ee9333
Jul 18 13:32:31.795766 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:31.795740    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8926567829fff2679ba8a397c742012f.slice/crio-57acd8b89703713c31f9366bc22f40921d0045a1cae1104a53281c9262f679b5 WatchSource:0}: Error finding container 57acd8b89703713c31f9366bc22f40921d0045a1cae1104a53281c9262f679b5: Status 404 returned error can't find the container with id 57acd8b89703713c31f9366bc22f40921d0045a1cae1104a53281c9262f679b5
Jul 18 13:32:31.801450 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.801432    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 13:32:31.850305 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.850277    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:31.859705 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.859672    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:31.927157 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:31.927128    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:31.961338 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:31.961311    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:32.061600 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.061560    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found"
Jul 18 13:32:32.132588 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.132531    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:32.175894 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.175860    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:32.186151 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.186125    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 13:32:32.187093 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.187079    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal"
Jul 18 13:32:32.197265 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.197249    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 13:32:32.654344 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.654309    2577 apiserver.go:52] "Watching apiserver"
Jul 18 13:32:32.665870 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.665830    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 13:32:32.667585 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.667552    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-j5mqw","openshift-network-diagnostics/network-check-target-xkxjc","openshift-network-operator/iptables-alerter-hqqr4","openshift-ovn-kubernetes/ovnkube-node-jmrzc","kube-system/global-pull-secret-syncer-2849v","kube-system/konnectivity-agent-6dn7v","openshift-dns/node-resolver-9fnb7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal","openshift-multus/multus-7nltq","openshift-multus/network-metrics-daemon-5pfd2","kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r","openshift-cluster-node-tuning-operator/tuned-jstns","openshift-image-registry/node-ca-4v2rl"]
Jul 18 13:32:32.670200 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.669725    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.671720 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.671667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.672506 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.672345    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ms952\""
Jul 18 13:32:32.672506 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.672384    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.672776 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.672727    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.674014 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.673989    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.674196 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.674182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.674196 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.674188    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.674337 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.674302    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4c2sp\""
Jul 18 13:32:32.677390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.675193    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 13:32:32.677390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.676227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.677390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.676678    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 13:32:32.677390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.677079    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vjh78\""
Jul 18 13:32:32.677390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.677194    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.677390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.677289    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.679246 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.678549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.679246 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.678615    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:32.679246 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.678940    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 13:32:32.679246 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.678988    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 13:32:32.679847 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.679822    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.679964 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.679756    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.682927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.680654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tbnc6\""
Jul 18 13:32:32.682927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.681134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 13:32:32.682927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.681763    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 13:32:32.682927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.682382    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.684457 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.684396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.684626 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.684594    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 13:32:32.686019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.685779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vttz2\""
Jul 18 13:32:32.686019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.685802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 13:32:32.687032 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.686955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.687402 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.687384    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.687786 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.687769    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 13:32:32.687994 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.687975    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.688125 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.687865    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 13:32:32.688207 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.687771    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2l6r4\""
Jul 18 13:32:32.689139 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.688980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-modprobe-d\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689139 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-tuned\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689139 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-run-netns\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689139 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-etc-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-env-overrides\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysctl-d\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-lib-modules\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-kubelet\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-systemd-units\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-systemd\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-tmp-dir\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.689388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-systemd\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-log-socket\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-sys\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-host\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18afb7d6-073a-469e-baac-9abcb1ab264e-tmp\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-slash\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4jnw\" (UniqueName: \"kubernetes.io/projected/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-kube-api-access-n4jnw\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-kubernetes\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.689735 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysctl-conf\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-var-lib-kubelet\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-var-lib-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689810    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-ovnkube-config\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.689878    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-run\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffc7c024-fafe-4dbc-b667-5761814bc1f4-iptables-alerter-script\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.689999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-ovn\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-cni-netd\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-ovnkube-script-lib\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysconfig\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzkzw\" (UniqueName: \"kubernetes.io/projected/18afb7d6-073a-469e-baac-9abcb1ab264e-kube-api-access-dzkzw\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6pvk\" (UniqueName: \"kubernetes.io/projected/ffc7c024-fafe-4dbc-b667-5761814bc1f4-kube-api-access-n6pvk\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngp4r\" (UniqueName: \"kubernetes.io/projected/3b02ccd9-1bb3-44fc-861a-42805558947a-kube-api-access-ngp4r\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-hosts-file\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffc7c024-fafe-4dbc-b667-5761814bc1f4-host-slash\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-node-log\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-cni-bin\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b02ccd9-1bb3-44fc-861a-42805558947a-ovn-node-metrics-cert\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690654    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 13:32:32.690770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690733    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jxj27\""
Jul 18 13:32:32.691395 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.690995    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 13:32:32.691918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.691887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:32.692006 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.691974    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:32.692944 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.692923    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 13:32:32.694388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.694367    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.696651 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.696632    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.696723 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.696693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.697117 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.697094    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 13:32:32.697207 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.697129    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.697207 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.697167    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lw8h8\""
Jul 18 13:32:32.699223 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.699204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 13:32:32.699325 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.699236    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 13:32:32.699325 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.699264    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lhpm5\""
Jul 18 13:32:32.699325 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.699238    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 13:32:32.705468 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.705445    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2b2w"
Jul 18 13:32:32.714919 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.714901    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2b2w"
Jul 18 13:32:32.776738 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.776709    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 13:32:32.791171 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysctl-conf\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-os-release\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-netns\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-run\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-cni-netd\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-hostroot\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cb6a15d-8760-41f7-a007-d22df913e121-dbus\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.791332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb9103a2-100d-4187-91fc-8708bbf5cd35-host\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysctl-conf\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6pvk\" (UniqueName: \"kubernetes.io/projected/ffc7c024-fafe-4dbc-b667-5761814bc1f4-kube-api-access-n6pvk\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngp4r\" (UniqueName: \"kubernetes.io/projected/3b02ccd9-1bb3-44fc-861a-42805558947a-kube-api-access-ngp4r\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-run\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-daemon-config\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-cni-netd\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-hosts-file\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-cni-bin\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b02ccd9-1bb3-44fc-861a-42805558947a-ovn-node-metrics-cert\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-cni-bin\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-hosts-file\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791615    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982xq\" (UniqueName: \"kubernetes.io/projected/8bb5079e-d591-4460-9b9b-6d508caf5620-kube-api-access-982xq\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791675    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.791754 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-os-release\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysctl-d\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-lib-modules\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-systemd\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-cni-multus\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791918    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-systemd\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysctl-d\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-lib-modules\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.791984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-tmp-dir\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-systemd\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-log-socket\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.792572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-cni-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-systemd\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-socket-dir-parent\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtk4d\" (UniqueName: \"kubernetes.io/projected/07af34cb-f6f9-4dd5-b7b2-038a374d8201-kube-api-access-jtk4d\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-log-socket\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-sys\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-host\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18afb7d6-073a-469e-baac-9abcb1ab264e-tmp\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-slash\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792332    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-sys\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-system-cni-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-host\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-slash\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792384    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-tmp-dir\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-multus-certs\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4c6ff44-14b9-4aab-9861-110804401955-agent-certs\") pod \"konnectivity-agent-6dn7v\" (UID: \"f4c6ff44-14b9-4aab-9861-110804401955\") " pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-var-lib-kubelet\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.793433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-var-lib-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-ovnkube-config\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-var-lib-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07af34cb-f6f9-4dd5-b7b2-038a374d8201-cni-binary-copy\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792628    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-var-lib-kubelet\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-conf-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4c6ff44-14b9-4aab-9861-110804401955-konnectivity-ca\") pod \"konnectivity-agent-6dn7v\" (UID: \"f4c6ff44-14b9-4aab-9861-110804401955\") " pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-system-cni-dir\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffc7c024-fafe-4dbc-b667-5761814bc1f4-iptables-alerter-script\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-ovn\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-ovnkube-script-lib\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-kubelet\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-run-ovn\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwx65\" (UniqueName: \"kubernetes.io/projected/cb9103a2-100d-4187-91fc-8708bbf5cd35-kube-api-access-vwx65\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.794355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysconfig\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.792982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzkzw\" (UniqueName: \"kubernetes.io/projected/18afb7d6-073a-469e-baac-9abcb1ab264e-kube-api-access-dzkzw\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-cnibin\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffc7c024-fafe-4dbc-b667-5761814bc1f4-host-slash\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-node-log\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793134    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-modprobe-d\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-ovnkube-config\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-tuned\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-run-netns\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-etc-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-env-overrides\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-k8s-cni-cncf-io\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-kubelet\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffc7c024-fafe-4dbc-b667-5761814bc1f4-iptables-alerter-script\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-systemd-units\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.795309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-ovnkube-script-lib\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-cni-bin\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cb6a15d-8760-41f7-a007-d22df913e121-kubelet-config\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-systemd-units\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9hw5\" (UniqueName: \"kubernetes.io/projected/d414579c-495e-4311-9ab8-bc2bae36afd9-kube-api-access-q9hw5\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-run-netns\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-etc-kubernetes\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-kubelet\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtr84\" (UniqueName: \"kubernetes.io/projected/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-kube-api-access-vtr84\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-etc-openvswitch\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-cnibin\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-sysconfig\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-modprobe-d\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b02ccd9-1bb3-44fc-861a-42805558947a-node-log\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b02ccd9-1bb3-44fc-861a-42805558947a-env-overrides\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.796136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffc7c024-fafe-4dbc-b667-5761814bc1f4-host-slash\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-device-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cb9103a2-100d-4187-91fc-8708bbf5cd35-serviceca\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4jnw\" (UniqueName: \"kubernetes.io/projected/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-kube-api-access-n4jnw\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-kubernetes\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.793946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-kubernetes\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.796145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18afb7d6-073a-469e-baac-9abcb1ab264e-etc-tuned\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.796205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18afb7d6-073a-469e-baac-9abcb1ab264e-tmp\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.796911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.796304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b02ccd9-1bb3-44fc-861a-42805558947a-ovn-node-metrics-cert\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.799697 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.799475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngp4r\" (UniqueName: \"kubernetes.io/projected/3b02ccd9-1bb3-44fc-861a-42805558947a-kube-api-access-ngp4r\") pod \"ovnkube-node-jmrzc\" (UID: \"3b02ccd9-1bb3-44fc-861a-42805558947a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:32.799697 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.799618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6pvk\" (UniqueName: \"kubernetes.io/projected/ffc7c024-fafe-4dbc-b667-5761814bc1f4-kube-api-access-n6pvk\") pod \"iptables-alerter-hqqr4\" (UID: \"ffc7c024-fafe-4dbc-b667-5761814bc1f4\") " pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:32.800690 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.800645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" event={"ID":"da14a40c1d1653f9f035a1d464909031","Type":"ContainerStarted","Data":"a640bffe5076b6902b0ad549c3d41e8521c7321f09373341949bde5bc0ee9333"}
Jul 18 13:32:32.801891 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.801827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" event={"ID":"8926567829fff2679ba8a397c742012f","Type":"ContainerStarted","Data":"57acd8b89703713c31f9366bc22f40921d0045a1cae1104a53281c9262f679b5"}
Jul 18 13:32:32.803766 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.803743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzkzw\" (UniqueName: \"kubernetes.io/projected/18afb7d6-073a-469e-baac-9abcb1ab264e-kube-api-access-dzkzw\") pod \"tuned-jstns\" (UID: \"18afb7d6-073a-469e-baac-9abcb1ab264e\") " pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:32.804680 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.804662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4jnw\" (UniqueName: \"kubernetes.io/projected/9dd4addb-d362-4a5a-96ff-4bc28f94e2f3-kube-api-access-n4jnw\") pod \"node-resolver-9fnb7\" (UID: \"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3\") " pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.862150 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.862102    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 13:32:32.894676 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-cni-bin\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894676 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cb6a15d-8760-41f7-a007-d22df913e121-kubelet-config\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9hw5\" (UniqueName: \"kubernetes.io/projected/d414579c-495e-4311-9ab8-bc2bae36afd9-kube-api-access-q9hw5\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-etc-kubernetes\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtr84\" (UniqueName: \"kubernetes.io/projected/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-kube-api-access-vtr84\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-cni-bin\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-cnibin\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-device-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cb6a15d-8760-41f7-a007-d22df913e121-kubelet-config\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-etc-kubernetes\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cb9103a2-100d-4187-91fc-8708bbf5cd35-serviceca\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-os-release\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-netns\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-hostroot\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.894906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cb6a15d-8760-41f7-a007-d22df913e121-dbus\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb9103a2-100d-4187-91fc-8708bbf5cd35-host\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.894998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-daemon-config\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982xq\" (UniqueName: \"kubernetes.io/projected/8bb5079e-d591-4460-9b9b-6d508caf5620-kube-api-access-982xq\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895285    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-os-release\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cb9103a2-100d-4187-91fc-8708bbf5cd35-serviceca\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-netns\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-hostroot\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-os-release\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-device-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-cni-multus\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.895454    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:32.895613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.895537    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:33.395499112 +0000 UTC m=+3.052189223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-cni-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-socket-dir-parent\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtk4d\" (UniqueName: \"kubernetes.io/projected/07af34cb-f6f9-4dd5-b7b2-038a374d8201-kube-api-access-jtk4d\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-system-cni-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-multus-certs\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4c6ff44-14b9-4aab-9861-110804401955-agent-certs\") pod \"konnectivity-agent-6dn7v\" (UID: \"f4c6ff44-14b9-4aab-9861-110804401955\") " pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07af34cb-f6f9-4dd5-b7b2-038a374d8201-cni-binary-copy\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-conf-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4c6ff44-14b9-4aab-9861-110804401955-konnectivity-ca\") pod \"konnectivity-agent-6dn7v\" (UID: \"f4c6ff44-14b9-4aab-9861-110804401955\") " pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-system-cni-dir\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-kubelet\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.896601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwx65\" (UniqueName: \"kubernetes.io/projected/cb9103a2-100d-4187-91fc-8708bbf5cd35-kube-api-access-vwx65\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-cnibin\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-daemon-config\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.895981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-cnibin\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cb6a15d-8760-41f7-a007-d22df913e121-dbus\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb9103a2-100d-4187-91fc-8708bbf5cd35-host\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-os-release\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-k8s-cni-cncf-io\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-system-cni-dir\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-cni-multus\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.896493    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-k8s-cni-cncf-io\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.897451 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-run-multus-certs\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bb5079e-d591-4460-9b9b-6d508caf5620-cnibin\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.896604    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:33.396585763 +0000 UTC m=+3.053275888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-cni-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-socket-dir-parent\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-multus-conf-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-host-var-lib-kubelet\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.896966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.897145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07af34cb-f6f9-4dd5-b7b2-038a374d8201-system-cni-dir\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.897181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07af34cb-f6f9-4dd5-b7b2-038a374d8201-cni-binary-copy\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.897415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4c6ff44-14b9-4aab-9861-110804401955-konnectivity-ca\") pod \"konnectivity-agent-6dn7v\" (UID: \"f4c6ff44-14b9-4aab-9861-110804401955\") " pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.898427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.898193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bb5079e-d591-4460-9b9b-6d508caf5620-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.899405 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.899388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4c6ff44-14b9-4aab-9861-110804401955-agent-certs\") pod \"konnectivity-agent-6dn7v\" (UID: \"f4c6ff44-14b9-4aab-9861-110804401955\") " pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:32.903707 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.903681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9hw5\" (UniqueName: \"kubernetes.io/projected/d414579c-495e-4311-9ab8-bc2bae36afd9-kube-api-access-q9hw5\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:32.904705 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.904649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtr84\" (UniqueName: \"kubernetes.io/projected/1814189d-cd7f-4b41-9d0e-8e788dd22bd5-kube-api-access-vtr84\") pod \"aws-ebs-csi-driver-node-vm97r\" (UID: \"1814189d-cd7f-4b41-9d0e-8e788dd22bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:32.904705 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.904686    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:32.904705 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.904703    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:32.904933 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.904713    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:32.904933 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:32.904775    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:33.404760349 +0000 UTC m=+3.061450475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:32.904933 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.904856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-982xq\" (UniqueName: \"kubernetes.io/projected/8bb5079e-d591-4460-9b9b-6d508caf5620-kube-api-access-982xq\") pod \"multus-additional-cni-plugins-j5mqw\" (UID: \"8bb5079e-d591-4460-9b9b-6d508caf5620\") " pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:32.907624 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.907601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtk4d\" (UniqueName: \"kubernetes.io/projected/07af34cb-f6f9-4dd5-b7b2-038a374d8201-kube-api-access-jtk4d\") pod \"multus-7nltq\" (UID: \"07af34cb-f6f9-4dd5-b7b2-038a374d8201\") " pod="openshift-multus/multus-7nltq"
Jul 18 13:32:32.907951 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.907934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwx65\" (UniqueName: \"kubernetes.io/projected/cb9103a2-100d-4187-91fc-8708bbf5cd35-kube-api-access-vwx65\") pod \"node-ca-4v2rl\" (UID: \"cb9103a2-100d-4187-91fc-8708bbf5cd35\") " pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:32.982829 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.982800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9fnb7"
Jul 18 13:32:32.993648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:32.993627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jstns"
Jul 18 13:32:33.000528 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.000508    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hqqr4"
Jul 18 13:32:33.007259 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.007233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:32:33.013219 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.013200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:33.018982 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.018961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7nltq"
Jul 18 13:32:33.024764 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.024745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5mqw"
Jul 18 13:32:33.031415 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.031392    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4v2rl"
Jul 18 13:32:33.036229 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.036214    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r"
Jul 18 13:32:33.400954 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.400922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:33.401112 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.400981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:33.401112 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.401095    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:33.401112 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.401113    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:33.401240 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.401152    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:34.401131705 +0000 UTC m=+4.057821813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:33.401240 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.401181    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:34.401160352 +0000 UTC m=+4.057850485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:33.501532 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.501489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:33.501707 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.501662    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:33.501707 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.501685    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:33.501707 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.501699    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:33.501848 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.501759    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:34.501743895 +0000 UTC m=+4.158434008 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:33.718596 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.718488    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 13:27:33 +0000 UTC" deadline="2028-03-10 02:50:01.167816612 +0000 UTC"
Jul 18 13:32:33.718596 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.718547    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14413h17m27.449273698s"
Jul 18 13:32:33.796388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.796359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:33.796557 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.796473    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:33.796557 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:33.796544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:33.796661 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:33.796642    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:33.823460 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:33.823398    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1814189d_cd7f_4b41_9d0e_8e788dd22bd5.slice/crio-377cbb9517fd0ab841ef8b0815a0e4404efbf836742813665c65c8470c34356f WatchSource:0}: Error finding container 377cbb9517fd0ab841ef8b0815a0e4404efbf836742813665c65c8470c34356f: Status 404 returned error can't find the container with id 377cbb9517fd0ab841ef8b0815a0e4404efbf836742813665c65c8470c34356f
Jul 18 13:32:33.824678 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:33.824656    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc7c024_fafe_4dbc_b667_5761814bc1f4.slice/crio-11893fd8dd231f1ead849b4750f579a909e09a67158ea156481048a508979789 WatchSource:0}: Error finding container 11893fd8dd231f1ead849b4750f579a909e09a67158ea156481048a508979789: Status 404 returned error can't find the container with id 11893fd8dd231f1ead849b4750f579a909e09a67158ea156481048a508979789
Jul 18 13:32:33.827611 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:33.827323    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b02ccd9_1bb3_44fc_861a_42805558947a.slice/crio-d4ea55ccb60b9075c9e7ddfd1c40c875bfd3d7a49d9ba5d63956517d4c62ce6a WatchSource:0}: Error finding container d4ea55ccb60b9075c9e7ddfd1c40c875bfd3d7a49d9ba5d63956517d4c62ce6a: Status 404 returned error can't find the container with id d4ea55ccb60b9075c9e7ddfd1c40c875bfd3d7a49d9ba5d63956517d4c62ce6a
Jul 18 13:32:33.829905 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:33.829813    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb5079e_d591_4460_9b9b_6d508caf5620.slice/crio-3f48adff33f91b4927f1abbde96dc06864aec239bcc823cf0bede53ad36504a0 WatchSource:0}: Error finding container 3f48adff33f91b4927f1abbde96dc06864aec239bcc823cf0bede53ad36504a0: Status 404 returned error can't find the container with id 3f48adff33f91b4927f1abbde96dc06864aec239bcc823cf0bede53ad36504a0
Jul 18 13:32:33.831394 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:33.831364    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9103a2_100d_4187_91fc_8708bbf5cd35.slice/crio-509ab65fb953d1cc565e0ce8a5c245cea5d6b62146406272af7d024d95852e0b WatchSource:0}: Error finding container 509ab65fb953d1cc565e0ce8a5c245cea5d6b62146406272af7d024d95852e0b: Status 404 returned error can't find the container with id 509ab65fb953d1cc565e0ce8a5c245cea5d6b62146406272af7d024d95852e0b
Jul 18 13:32:33.832535 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:32:33.832356    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18afb7d6_073a_469e_baac_9abcb1ab264e.slice/crio-77b4d55ac5e95c6c3dc6611e03cec1c205d00feabc01c26892442f14dd766942 WatchSource:0}: Error finding container 77b4d55ac5e95c6c3dc6611e03cec1c205d00feabc01c26892442f14dd766942: Status 404 returned error can't find the container with id 77b4d55ac5e95c6c3dc6611e03cec1c205d00feabc01c26892442f14dd766942
Jul 18 13:32:34.409027 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.408706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:34.409027 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.409021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:34.409304 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.409185    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:34.409304 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.409250    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:36.409228863 +0000 UTC m=+6.065918975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:34.409660 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.409639    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:34.409759 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.409699    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:36.409681707 +0000 UTC m=+6.066371822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:34.510487 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.510447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:34.510717 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.510693    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:34.510792 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.510726    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:34.510792 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.510740    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:34.510889 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.510840    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:36.510783335 +0000 UTC m=+6.167473449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:34.720282 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.720179    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 13:27:33 +0000 UTC" deadline="2027-12-21 11:58:11.833391067 +0000 UTC"
Jul 18 13:32:34.720282 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.720223    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12502h25m37.113173183s"
Jul 18 13:32:34.799922 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.799467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:34.799922 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:34.799657    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:34.808489 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.808458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9fnb7" event={"ID":"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3","Type":"ContainerStarted","Data":"249a31aa53a04e25257e516cbf71626f5218aa61c33312943f7e5a65be56c9cc"}
Jul 18 13:32:34.819937 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.819828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7nltq" event={"ID":"07af34cb-f6f9-4dd5-b7b2-038a374d8201","Type":"ContainerStarted","Data":"20036acc5b47101a9b2d63b65f21c3fe57fa99e49192a8eaa4de2784587f0bed"}
Jul 18 13:32:34.823228 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.823202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4v2rl" event={"ID":"cb9103a2-100d-4187-91fc-8708bbf5cd35","Type":"ContainerStarted","Data":"509ab65fb953d1cc565e0ce8a5c245cea5d6b62146406272af7d024d95852e0b"}
Jul 18 13:32:34.841683 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.841650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jstns" event={"ID":"18afb7d6-073a-469e-baac-9abcb1ab264e","Type":"ContainerStarted","Data":"77b4d55ac5e95c6c3dc6611e03cec1c205d00feabc01c26892442f14dd766942"}
Jul 18 13:32:34.846741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.846676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerStarted","Data":"3f48adff33f91b4927f1abbde96dc06864aec239bcc823cf0bede53ad36504a0"}
Jul 18 13:32:34.865497 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.865464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" event={"ID":"8926567829fff2679ba8a397c742012f","Type":"ContainerStarted","Data":"d28357c9164f4ccb96aa0b52e0eb7ec76b700b80e3255d303eb8b9d9d47dd191"}
Jul 18 13:32:34.873799 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.873767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6dn7v" event={"ID":"f4c6ff44-14b9-4aab-9861-110804401955","Type":"ContainerStarted","Data":"660035d74019b3c1fb773907aee3c63e6529f44c2b42ba98906f56f5533ec436"}
Jul 18 13:32:34.880157 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.880089    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" podStartSLOduration=1.880071024 podStartE2EDuration="1.880071024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:34.879202496 +0000 UTC m=+4.535892626" watchObservedRunningTime="2026-07-18 13:32:34.880071024 +0000 UTC m=+4.536761147"
Jul 18 13:32:34.883998 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.883943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"d4ea55ccb60b9075c9e7ddfd1c40c875bfd3d7a49d9ba5d63956517d4c62ce6a"}
Jul 18 13:32:34.899905 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.898191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hqqr4" event={"ID":"ffc7c024-fafe-4dbc-b667-5761814bc1f4","Type":"ContainerStarted","Data":"11893fd8dd231f1ead849b4750f579a909e09a67158ea156481048a508979789"}
Jul 18 13:32:34.900475 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.900446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r" event={"ID":"1814189d-cd7f-4b41-9d0e-8e788dd22bd5","Type":"ContainerStarted","Data":"377cbb9517fd0ab841ef8b0815a0e4404efbf836742813665c65c8470c34356f"}
Jul 18 13:32:34.903352 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.903322    2577 generic.go:356] "Generic (PLEG): container finished" podID="da14a40c1d1653f9f035a1d464909031" containerID="91cd2922b3026de15e39f1fcd00e7a1d74646c72ce6b168314db2db659adc24a" exitCode=0
Jul 18 13:32:34.903458 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:34.903365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" event={"ID":"da14a40c1d1653f9f035a1d464909031","Type":"ContainerDied","Data":"91cd2922b3026de15e39f1fcd00e7a1d74646c72ce6b168314db2db659adc24a"}
Jul 18 13:32:35.796070 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:35.796010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:35.796564 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:35.796184    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:35.796637 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:35.796601    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:35.796778 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:35.796692    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:35.928629 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:35.928494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" event={"ID":"da14a40c1d1653f9f035a1d464909031","Type":"ContainerStarted","Data":"5d421c6e5ab3b91ab3691d9af43cc93e4bfb4dd3d64bf3ae7a8b69a95340c598"}
Jul 18 13:32:36.429237 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:36.429187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:36.429452 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:36.429303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:36.429452 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.429440    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:36.429581 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.429509    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:40.429487945 +0000 UTC m=+10.086178061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:36.429581 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.429574    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:36.429754 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.429610    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:40.429597807 +0000 UTC m=+10.086287922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:36.530532 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:36.530492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:36.530728 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.530708    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:36.530789 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.530728    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:36.530789 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.530743    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:36.530897 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.530804    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:40.530785473 +0000 UTC m=+10.187475585 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:36.799644 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:36.798664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:36.799644 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:36.798810    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:37.796202 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:37.796169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:37.796397 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:37.796323    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:37.796397 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:37.796361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:37.796527 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:37.796484    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:38.796668 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:38.796629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:38.797168 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:38.796795    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:39.796622 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:39.796580    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:39.796840 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:39.796625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:39.796840 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:39.796734    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:39.797317 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:39.797217    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:40.470272 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:40.470214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:40.470473 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:40.470301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:40.470473 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.470401    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:40.470473 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.470435    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:40.470644 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.470479    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:48.470454355 +0000 UTC m=+18.127144481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:40.470644 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.470500    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:48.470489651 +0000 UTC m=+18.127179775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:40.571294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:40.571177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:40.571466 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.571367    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:40.571466 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.571394    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:40.571466 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.571404    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:40.571466 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.571461    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:32:48.571445624 +0000 UTC m=+18.228135738 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:40.798032 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:40.797594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:40.798032 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:40.797730    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:41.796833 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:41.796755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:41.796833 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:41.796784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:41.797068 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:41.796908    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:41.797068 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:41.797016    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:42.796154 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:42.796108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:42.796624 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:42.796305    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:43.795869 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:43.795826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:43.796092 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:43.795828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:43.796092 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:43.795954    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:43.796211 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:43.796102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:44.796675 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:44.796639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:44.797148 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:44.796774    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:45.795835 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:45.795803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:45.795994 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:45.795937    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:45.795994 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:45.795971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:45.796141 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:45.796034    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:46.796247 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:46.796211    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:46.796643 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:46.796365    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:47.796640 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:47.796606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:47.797189 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:47.796733    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:47.797189 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:47.796793    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:47.797189 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:47.796892    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:48.531647 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:48.531614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:48.531811 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:48.531677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:48.531811 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.531777    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:48.531992 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.531836    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:04.531817244 +0000 UTC m=+34.188507365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:32:48.531992 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.531885    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:48.531992 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.531944    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:04.531928358 +0000 UTC m=+34.188618465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:32:48.632182 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:48.632148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:48.632344 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.632322    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:32:48.632344 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.632341    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:32:48.632432 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.632352    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:48.632432 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.632409    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:04.632390397 +0000 UTC m=+34.289080524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:32:48.799580 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:48.799506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:48.799965 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:48.799632    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:49.795946 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:49.795911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:49.796169 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:49.796063    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:49.796169 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:49.796113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:49.796276 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:49.796242    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:50.797466 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:50.797201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:50.797466 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:50.797312    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:51.796272 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:51.796241    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:51.796440 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:51.796245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:51.796440 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:51.796356    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:51.796440 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:51.796418    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:52.796788 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:52.796756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:52.797224 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:52.796887    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:53.796226 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.796074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:53.796383 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.796098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:53.796383 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:53.796324    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:53.796528 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:53.796389    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:53.967343 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.967164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jstns" event={"ID":"18afb7d6-073a-469e-baac-9abcb1ab264e","Type":"ContainerStarted","Data":"be63e10513453a4ee67fdcf13ddb335d4f62b4e1311509d7724ef4a708ed6b3a"}
Jul 18 13:32:53.968578 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.968557    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb5079e-d591-4460-9b9b-6d508caf5620" containerID="ecc75b564a4993a3ba5987e4ea320e2bfc3efb16af53378c21baaf2ed0fdca1d" exitCode=0
Jul 18 13:32:53.968682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.968624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerDied","Data":"ecc75b564a4993a3ba5987e4ea320e2bfc3efb16af53378c21baaf2ed0fdca1d"}
Jul 18 13:32:53.970172 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.970129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6dn7v" event={"ID":"f4c6ff44-14b9-4aab-9861-110804401955","Type":"ContainerStarted","Data":"d0ebbc5d3a316da17660140ca4ff35b318113cebc3bf63366e46026e4e7cb626"}
Jul 18 13:32:53.972412 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.972393    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:32:53.972751 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.972728    2577 generic.go:356] "Generic (PLEG): container finished" podID="3b02ccd9-1bb3-44fc-861a-42805558947a" containerID="bdbe171d30845b2c50d002e7617ffad4a05884017bb599db88a069d470ac8abf" exitCode=1
Jul 18 13:32:53.972835 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.972799    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"8400c32fc2f12b060a2408018181029e744162c996450477ed0b2e3ca943dc44"}
Jul 18 13:32:53.972895 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.972874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"efcc146965de493321bf6f5cf9345a9aa9219bfe8f96048cf7d9a1316b3a7d9a"}
Jul 18 13:32:53.972948 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.972917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerDied","Data":"bdbe171d30845b2c50d002e7617ffad4a05884017bb599db88a069d470ac8abf"}
Jul 18 13:32:53.972999 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.972964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"820d5cfd9454e28cf93e0706864304cf0d77b6d0c76e34a96b0299460a284908"}
Jul 18 13:32:53.974305 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.974280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r" event={"ID":"1814189d-cd7f-4b41-9d0e-8e788dd22bd5","Type":"ContainerStarted","Data":"9c6ef26b8eae5b51ef09df35372710d22bb039a714dcb7a4fa5f089172faecb3"}
Jul 18 13:32:53.977617 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.977588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9fnb7" event={"ID":"9dd4addb-d362-4a5a-96ff-4bc28f94e2f3","Type":"ContainerStarted","Data":"d05583a2e1708dff611e614b29b785a9bd34b5c9d211e70748eb3065b16cab16"}
Jul 18 13:32:53.978998 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.978977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7nltq" event={"ID":"07af34cb-f6f9-4dd5-b7b2-038a374d8201","Type":"ContainerStarted","Data":"a1306ea69aab02bbe20b441c3999d41512df4674d461ab0fbf47301968524a2d"}
Jul 18 13:32:53.980182 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.980166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4v2rl" event={"ID":"cb9103a2-100d-4187-91fc-8708bbf5cd35","Type":"ContainerStarted","Data":"86a89da850d442124f0f869c18133f515d843b2a27f618525ff197b518fe5c3d"}
Jul 18 13:32:53.981966 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.981925    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jstns" podStartSLOduration=3.419227864 podStartE2EDuration="22.981909974s" totalImagesPullingTime="19.56268211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:53.981876239 +0000 UTC m=+23.638566369" watchObservedRunningTime="2026-07-18 13:32:53.981909974 +0000 UTC m=+23.638600103"
Jul 18 13:32:53.982409 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.982387    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" podStartSLOduration=20.982381081 podStartE2EDuration="20.982381081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:35.944403395 +0000 UTC m=+5.601093528" watchObservedRunningTime="2026-07-18 13:32:53.982381081 +0000 UTC m=+23.639071209"
Jul 18 13:32:53.996237 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:53.996204    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6dn7v" podStartSLOduration=11.571014005 podStartE2EDuration="22.996192914s" totalImagesPullingTime="11.425178909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:53.996139189 +0000 UTC m=+23.652829318" watchObservedRunningTime="2026-07-18 13:32:53.996192914 +0000 UTC m=+23.652883043"
Jul 18 13:32:54.008453 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.008424    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4v2rl" podStartSLOduration=3.465779919 podStartE2EDuration="23.008413824s" totalImagesPullingTime="19.542633905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:54.008232944 +0000 UTC m=+23.664923072" watchObservedRunningTime="2026-07-18 13:32:54.008413824 +0000 UTC m=+23.665103954"
Jul 18 13:32:54.021366 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.021320    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9fnb7" podStartSLOduration=3.482440052 podStartE2EDuration="23.021305022s" totalImagesPullingTime="19.53886497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:54.021281652 +0000 UTC m=+23.677971780" watchObservedRunningTime="2026-07-18 13:32:54.021305022 +0000 UTC m=+23.677995152"
Jul 18 13:32:54.038423 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.038393    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7nltq" podStartSLOduration=3.438885247 podStartE2EDuration="23.038382279s" totalImagesPullingTime="19.599497032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:54.038329871 +0000 UTC m=+23.695019999" watchObservedRunningTime="2026-07-18 13:32:54.038382279 +0000 UTC m=+23.695072402"
Jul 18 13:32:54.126834 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.126755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.177327 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.177294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.271796 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.271765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.322126 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.322096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.605802 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.605772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.656344 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.656325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:54.796518 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.796305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:54.796518 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:54.796443    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:54.951461 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.951437    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 13:32:54.985778 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.985755    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:32:54.986544 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.986519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"690e5d6804d07018fceccaf514ea8e92fc4d336233dbba1243513a37615b8500"}
Jul 18 13:32:54.986658 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.986597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"3e16fdec5989b41b207e74329c8cd7c0412af35e6176d3d9d0dff6c629b31459"}
Jul 18 13:32:54.988283 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.988257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hqqr4" event={"ID":"ffc7c024-fafe-4dbc-b667-5761814bc1f4","Type":"ContainerStarted","Data":"8378137c7b12dde3f032aca2c28ea4ddc443aa0f4ef2a79b594b0a40186ea465"}
Jul 18 13:32:54.992800 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:54.990874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r" event={"ID":"1814189d-cd7f-4b41-9d0e-8e788dd22bd5","Type":"ContainerStarted","Data":"794a723a9c5dffe5131ade0aa0260704238e8964cc075a8168563723a9211d13"}
Jul 18 13:32:55.394151 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.394109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:55.443622 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.443588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:55.739959 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.739806    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T13:32:54.951456257Z","UUID":"917e2684-a8ab-4fe6-b268-b6c65d87c3b6","Handler":null,"Name":"","Endpoint":""}
Jul 18 13:32:55.741482 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.741460    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 13:32:55.741604 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.741500    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 13:32:55.794731 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.794696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:55.796726 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.796704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:55.796726 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.796722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:55.796895 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:55.796823    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:55.796950 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:55.796926    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:55.894762 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:55.894725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:56.176217 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:56.176184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:56.603325 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:56.603139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:56.796296 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:56.796265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:56.796480 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:56.796402    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:56.999459 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:56.999375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:56.999459 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:56.999429    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:32:56.999875 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:56.999851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"f9ad0f851326f9c2c3ba72dda836c6a684368be6823086a5899bf877a300c1fb"}
Jul 18 13:32:57.002322 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.002297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r" event={"ID":"1814189d-cd7f-4b41-9d0e-8e788dd22bd5","Type":"ContainerStarted","Data":"3a4e69eb26aaf3b84e90ad48a1d5e2918f3510289d16fd910689fc40554a67a4"}
Jul 18 13:32:57.018770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.018735    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vm97r" podStartSLOduration=4.12500305 podStartE2EDuration="26.018719076s" totalImagesPullingTime="21.893716026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:57.01865056 +0000 UTC m=+26.675340701" watchObservedRunningTime="2026-07-18 13:32:57.018719076 +0000 UTC m=+26.675409205"
Jul 18 13:32:57.019076 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.019050    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hqqr4" podStartSLOduration=6.470511587 podStartE2EDuration="26.019029053s" totalImagesPullingTime="19.548517466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:32:55.002323539 +0000 UTC m=+24.659013662" watchObservedRunningTime="2026-07-18 13:32:57.019029053 +0000 UTC m=+26.675719182"
Jul 18 13:32:57.100209 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.100176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:57.662232 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.662189    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:57.662788 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.662766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:57.796307 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.796268    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:57.796307 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:57.796307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:57.796517 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:57.796428    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:57.796567 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:57.796545    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:32:58.005053 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:58.004958    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:58.005598 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:58.005577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6dn7v"
Jul 18 13:32:58.057949 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:58.057919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:32:58.796288 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:58.796125    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:32:58.796855 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:58.796361    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:32:59.008513 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.008441    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb5079e-d591-4460-9b9b-6d508caf5620" containerID="b76599ce90313000cf1173c468a2d2ab203cfb0e5a1a72a90fc3fbbdbcdf7d79" exitCode=0
Jul 18 13:32:59.008667 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.008520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerDied","Data":"b76599ce90313000cf1173c468a2d2ab203cfb0e5a1a72a90fc3fbbdbcdf7d79"}
Jul 18 13:32:59.011853 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.011839    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:32:59.012268 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.012240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"eb06e42a93cd9e93735ec6c14e6b169e9082c8ed97e72b09e13873e0e8d0887b"}
Jul 18 13:32:59.012753 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.012737    2577 scope.go:117] "RemoveContainer" containerID="bdbe171d30845b2c50d002e7617ffad4a05884017bb599db88a069d470ac8abf"
Jul 18 13:32:59.796638 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.796610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:32:59.797136 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:59.796717    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:32:59.797136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:32:59.796767    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:32:59.797136 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:32:59.796859    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:33:00.018690 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.018670    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:33:00.019121 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.019089    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" event={"ID":"3b02ccd9-1bb3-44fc-861a-42805558947a","Type":"ContainerStarted","Data":"81a98a80871de79b57436c098043676fee6949ff8ee4519e18404fbc51f022a8"}
Jul 18 13:33:00.019578 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.019482    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:33:00.019578 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.019549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:33:00.019712 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.019604    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:33:00.038730 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.038639    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:33:00.039486 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.039458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:33:00.069309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.069243    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc" podStartSLOduration=9.453470378 podStartE2EDuration="29.06922219s" totalImagesPullingTime="19.615751812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:00.067464271 +0000 UTC m=+29.724154403" watchObservedRunningTime="2026-07-18 13:33:00.06922219 +0000 UTC m=+29.725912320"
Jul 18 13:33:00.313049 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.312853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5pfd2"]
Jul 18 13:33:00.313312 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.313140    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:00.313312 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:00.313265    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:33:00.315582 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.315563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xkxjc"]
Jul 18 13:33:00.315667 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.315631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:00.315712 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:00.315696    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:33:00.324091 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.324063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2849v"]
Jul 18 13:33:00.324191 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:00.324162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:00.324274 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:00.324255    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:33:01.023500 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:01.023463    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb5079e-d591-4460-9b9b-6d508caf5620" containerID="8a4b9d13eff8ac56c2bc5be3b0be7e9178d8e1f31263fb28fa57ae00dd4c9b73" exitCode=0
Jul 18 13:33:01.023927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:01.023546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerDied","Data":"8a4b9d13eff8ac56c2bc5be3b0be7e9178d8e1f31263fb28fa57ae00dd4c9b73"}
Jul 18 13:33:01.796475 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:01.796443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:01.796475 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:01.796463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:01.796664 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:01.796550    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:01.796664 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:01.796565    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:33:01.796664 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:01.796613    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:33:01.796761 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:01.796679    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:33:03.029558 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:03.029521    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb5079e-d591-4460-9b9b-6d508caf5620" containerID="0446220beb4e5767c637f15518cd59898f9cbcc9b440df6b34f6b91eb56ef8cd" exitCode=0
Jul 18 13:33:03.029558 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:03.029560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerDied","Data":"0446220beb4e5767c637f15518cd59898f9cbcc9b440df6b34f6b91eb56ef8cd"}
Jul 18 13:33:03.796387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:03.796352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:03.796387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:03.796383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:03.796750 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:03.796363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:03.796750 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:03.796498    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:33:03.796750 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:03.796624    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:33:03.796750 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:03.796701    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:33:04.550228 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:04.550190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:04.550699 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:04.550264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:04.550699 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.550324    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:33:04.550699 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.550381    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 13:33:04.550699 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.550389    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:36.550371952 +0000 UTC m=+66.207062061 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 13:33:04.550699 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.550430    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret podName:1cb6a15d-8760-41f7-a007-d22df913e121 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:36.550414562 +0000 UTC m=+66.207104679 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret") pod "global-pull-secret-syncer-2849v" (UID: "1cb6a15d-8760-41f7-a007-d22df913e121") : object "kube-system"/"original-pull-secret" not registered
Jul 18 13:33:04.651626 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:04.651594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:04.652218 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.651784    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 13:33:04.652218 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.651809    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 13:33:04.652218 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.651821    2577 projected.go:196] Error preparing data for projected volume kube-api-access-n5dck for pod openshift-network-diagnostics/network-check-target-xkxjc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:33:04.652218 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:04.651878    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck podName:37f62892-f870-401e-a292-40b202b6e566 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:36.651859254 +0000 UTC m=+66.308549377 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5dck" (UniqueName: "kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck") pod "network-check-target-xkxjc" (UID: "37f62892-f870-401e-a292-40b202b6e566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 13:33:05.796417 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:05.796388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:05.796841 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:05.796387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:05.796841 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:05.796487    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2849v" podUID="1cb6a15d-8760-41f7-a007-d22df913e121"
Jul 18 13:33:05.796841 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:05.796391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:05.796841 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:05.796603    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:33:05.796841 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:05.796690    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xkxjc" podUID="37f62892-f870-401e-a292-40b202b6e566"
Jul 18 13:33:05.945238 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:05.945185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:06.044908 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.044869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:06.159080 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.158987    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeReady"
Jul 18 13:33:06.159215 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.159147    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 13:33:06.199939 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.199904    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9j6vs"]
Jul 18 13:33:06.233940 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.233897    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7hw6q"]
Jul 18 13:33:06.234108 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.234054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.236225 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.236197    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 13:33:06.236362 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.236203    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 13:33:06.236614 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.236597    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t94bf\""
Jul 18 13:33:06.258297 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.258273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9j6vs"]
Jul 18 13:33:06.258407 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.258317    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7hw6q"]
Jul 18 13:33:06.258407 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.258385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.260681 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.260655    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 13:33:06.260787 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.260654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v6fm2\""
Jul 18 13:33:06.260787 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.260697    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 13:33:06.260787 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.260660    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 13:33:06.304985 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.304955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:06.360863 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.360825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:06.363822 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.363798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-config-volume\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.363822 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.363831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbhwt\" (UniqueName: \"kubernetes.io/projected/19c69f7e-d198-4022-abaf-cb8b8f3b329b-kube-api-access-fbhwt\") pod \"ingress-canary-7hw6q\" (UID: \"19c69f7e-d198-4022-abaf-cb8b8f3b329b\") " pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.364026 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.363865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c69f7e-d198-4022-abaf-cb8b8f3b329b-cert\") pod \"ingress-canary-7hw6q\" (UID: \"19c69f7e-d198-4022-abaf-cb8b8f3b329b\") " pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.364026 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.364000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-tmp-dir\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.364128 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.364023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5jpr\" (UniqueName: \"kubernetes.io/projected/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-kube-api-access-z5jpr\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.364128 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.364088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.465169 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-tmp-dir\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.465169 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5jpr\" (UniqueName: \"kubernetes.io/projected/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-kube-api-access-z5jpr\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.465169 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.465432 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-config-volume\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.465432 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbhwt\" (UniqueName: \"kubernetes.io/projected/19c69f7e-d198-4022-abaf-cb8b8f3b329b-kube-api-access-fbhwt\") pod \"ingress-canary-7hw6q\" (UID: \"19c69f7e-d198-4022-abaf-cb8b8f3b329b\") " pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.465432 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c69f7e-d198-4022-abaf-cb8b8f3b329b-cert\") pod \"ingress-canary-7hw6q\" (UID: \"19c69f7e-d198-4022-abaf-cb8b8f3b329b\") " pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.465432 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:06.465295    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:06.465432 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:06.465380    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:06.965359494 +0000 UTC m=+36.622049605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:06.465657 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465498    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-tmp-dir\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.465789 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.465769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-config-volume\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.469850 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.469702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c69f7e-d198-4022-abaf-cb8b8f3b329b-cert\") pod \"ingress-canary-7hw6q\" (UID: \"19c69f7e-d198-4022-abaf-cb8b8f3b329b\") " pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.474059 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.473990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5jpr\" (UniqueName: \"kubernetes.io/projected/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-kube-api-access-z5jpr\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.474161 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.474059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbhwt\" (UniqueName: \"kubernetes.io/projected/19c69f7e-d198-4022-abaf-cb8b8f3b329b-kube-api-access-fbhwt\") pod \"ingress-canary-7hw6q\" (UID: \"19c69f7e-d198-4022-abaf-cb8b8f3b329b\") " pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.567939 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.567909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7hw6q"
Jul 18 13:33:06.969913 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:06.969876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:06.970341 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:06.970033    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:06.970341 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:06.970118    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:07.970101031 +0000 UTC m=+37.626791138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:07.318980 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.318947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7hw6q"]
Jul 18 13:33:07.388139 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:33:07.388096    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c69f7e_d198_4022_abaf_cb8b8f3b329b.slice/crio-7bd75f698c111a6172324cf37fb076ce62e2f66e8f2018044938e88d2d0a4812 WatchSource:0}: Error finding container 7bd75f698c111a6172324cf37fb076ce62e2f66e8f2018044938e88d2d0a4812: Status 404 returned error can't find the container with id 7bd75f698c111a6172324cf37fb076ce62e2f66e8f2018044938e88d2d0a4812
Jul 18 13:33:07.796114 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.796090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:07.796252 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.796120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:07.796252 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.796088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:07.799030 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.799008    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 13:33:07.799161 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.799123    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l5j6b\""
Jul 18 13:33:07.800054 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.800016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 13:33:07.800054 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.800029    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 13:33:07.800179 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.800081    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfhtr\""
Jul 18 13:33:07.800332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.800316    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 13:33:07.978430 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:07.978402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:07.978868 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:07.978536    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:07.978868 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:07.978687    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:09.978661528 +0000 UTC m=+39.635351636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:08.045210 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:08.045172    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb5079e-d591-4460-9b9b-6d508caf5620" containerID="ecdc1149fa6a71aaf9b5852a669ac8e6d596a76c44160345b2bcfc7d795e7598" exitCode=0
Jul 18 13:33:08.045390 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:08.045249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerDied","Data":"ecdc1149fa6a71aaf9b5852a669ac8e6d596a76c44160345b2bcfc7d795e7598"}
Jul 18 13:33:08.046740 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:08.046665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7hw6q" event={"ID":"19c69f7e-d198-4022-abaf-cb8b8f3b329b","Type":"ContainerStarted","Data":"7bd75f698c111a6172324cf37fb076ce62e2f66e8f2018044938e88d2d0a4812"}
Jul 18 13:33:09.052843 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:09.052802    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb5079e-d591-4460-9b9b-6d508caf5620" containerID="db6b73bbc150472586b1872ef1ea2d54602224d51d3296515480321e7e3e99d9" exitCode=0
Jul 18 13:33:09.053323 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:09.052850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerDied","Data":"db6b73bbc150472586b1872ef1ea2d54602224d51d3296515480321e7e3e99d9"}
Jul 18 13:33:09.314050 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:09.313730    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:09.360071 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:09.360027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:09.996325 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:09.996272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:09.996464 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:09.996433    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:09.996502 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:09.996496    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:13.996480253 +0000 UTC m=+43.653170360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:10.057779 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.057749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" event={"ID":"8bb5079e-d591-4460-9b9b-6d508caf5620","Type":"ContainerStarted","Data":"30a64eeb5252d27f6cb0d21040854c86dea57670f1b054b3826f0c7806bbe7b6"}
Jul 18 13:33:10.059129 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.059109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7hw6q" event={"ID":"19c69f7e-d198-4022-abaf-cb8b8f3b329b","Type":"ContainerStarted","Data":"a6234c58c3286c02da780fffdca6f8c6a3988893620f4063774c5ad679e4ee46"}
Jul 18 13:33:10.084175 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.084131    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j5mqw" podStartSLOduration=8.825358283 podStartE2EDuration="39.084115157s" totalImagesPullingTime="30.258756874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:10.083849324 +0000 UTC m=+39.740539452" watchObservedRunningTime="2026-07-18 13:33:10.084115157 +0000 UTC m=+39.740805288"
Jul 18 13:33:10.100087 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.100020    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7hw6q" podStartSLOduration=2.027769014 podStartE2EDuration="4.100005546s" totalImagesPullingTime="2.072236532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:10.099672962 +0000 UTC m=+39.756363091" watchObservedRunningTime="2026-07-18 13:33:10.100005546 +0000 UTC m=+39.756695676"
Jul 18 13:33:10.181453 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.181395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:10.235939 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.235909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:10.373080 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.373049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:10.430868 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:10.430842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:14.021634 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:14.021599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:14.022002 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:14.021704    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:14.022002 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:14.021755    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:22.021741848 +0000 UTC m=+51.678431955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:15.318409 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:15.318379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:15.524545 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:15.524516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:22.072782 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:22.072744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:22.073414 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:22.072927    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:22.073414 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:22.073020    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:33:38.072993742 +0000 UTC m=+67.729683886 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:31.289701 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:31.289665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:31.337426 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:31.337401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:32.036427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:32.036404    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrzc"
Jul 18 13:33:32.143189 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:32.143153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:32.196494 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:32.196458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:33:36.565211 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.565169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:36.565790 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.565226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:33:36.567639 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.567607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 13:33:36.567797 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.567657    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 13:33:36.576492 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:36.576470    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 13:33:36.576566 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:36.576551    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:34:40.576526449 +0000 UTC m=+130.233216591 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : secret "metrics-daemon-secret" not found
Jul 18 13:33:36.580579 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.580558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cb6a15d-8760-41f7-a007-d22df913e121-original-pull-secret\") pod \"global-pull-secret-syncer-2849v\" (UID: \"1cb6a15d-8760-41f7-a007-d22df913e121\") " pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:36.616837 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.616808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2849v"
Jul 18 13:33:36.665973 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.665942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:36.668640 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.668597    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 13:33:36.678458 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.678426    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 13:33:36.689411 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.689387    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5dck\" (UniqueName: \"kubernetes.io/projected/37f62892-f870-401e-a292-40b202b6e566-kube-api-access-n5dck\") pod \"network-check-target-xkxjc\" (UID: \"37f62892-f870-401e-a292-40b202b6e566\") " pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:36.736902 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.736878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2849v"]
Jul 18 13:33:36.740745 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:33:36.740719    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb6a15d_8760_41f7_a007_d22df913e121.slice/crio-09a6927bc7341c67f48577a6e4cf6b80e94223d7565ae96e8ba4a200ba3f9512 WatchSource:0}: Error finding container 09a6927bc7341c67f48577a6e4cf6b80e94223d7565ae96e8ba4a200ba3f9512: Status 404 returned error can't find the container with id 09a6927bc7341c67f48577a6e4cf6b80e94223d7565ae96e8ba4a200ba3f9512
Jul 18 13:33:36.913529 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.913483    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfhtr\""
Jul 18 13:33:36.921319 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:36.921302    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:37.036168 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:37.036139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xkxjc"]
Jul 18 13:33:37.040028 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:33:37.040004    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f62892_f870_401e_a292_40b202b6e566.slice/crio-753870a475a133286747ef31ebe2309fb82c33c1589b74500390d561999c5fd2 WatchSource:0}: Error finding container 753870a475a133286747ef31ebe2309fb82c33c1589b74500390d561999c5fd2: Status 404 returned error can't find the container with id 753870a475a133286747ef31ebe2309fb82c33c1589b74500390d561999c5fd2
Jul 18 13:33:37.120975 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:37.120949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xkxjc" event={"ID":"37f62892-f870-401e-a292-40b202b6e566","Type":"ContainerStarted","Data":"753870a475a133286747ef31ebe2309fb82c33c1589b74500390d561999c5fd2"}
Jul 18 13:33:37.121997 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:37.121977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2849v" event={"ID":"1cb6a15d-8760-41f7-a007-d22df913e121","Type":"ContainerStarted","Data":"09a6927bc7341c67f48577a6e4cf6b80e94223d7565ae96e8ba4a200ba3f9512"}
Jul 18 13:33:38.074231 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:38.074192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:33:38.074683 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:38.074365    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:33:38.074683 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:33:38.074446    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:34:10.074424939 +0000 UTC m=+99.731115062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:33:42.137385 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:42.137347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xkxjc" event={"ID":"37f62892-f870-401e-a292-40b202b6e566","Type":"ContainerStarted","Data":"d3fc8f1035309e0549080be3ed66c9988413aba60e2902ab6a8ef50ef9e2fa7f"}
Jul 18 13:33:42.137841 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:42.137472    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:33:42.138903 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:42.138883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2849v" event={"ID":"1cb6a15d-8760-41f7-a007-d22df913e121","Type":"ContainerStarted","Data":"32c908637d308cd8049962ad1ed11c8c89d1793eb67a264bb76b9732dcc03fad"}
Jul 18 13:33:42.151238 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:42.151193    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xkxjc" podStartSLOduration=66.563729793 podStartE2EDuration="1m11.15118138s" totalImagesPullingTime="4.587451587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:42.150602896 +0000 UTC m=+71.807293025" watchObservedRunningTime="2026-07-18 13:33:42.15118138 +0000 UTC m=+71.807871509"
Jul 18 13:33:42.163070 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:33:42.163008    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2849v" podStartSLOduration=66.264642599 podStartE2EDuration="1m11.162996482s" totalImagesPullingTime="4.898353883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:33:42.162650094 +0000 UTC m=+71.819340233" watchObservedRunningTime="2026-07-18 13:33:42.162996482 +0000 UTC m=+71.819686612"
Jul 18 13:34:05.517704 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.517589    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"]
Jul 18 13:34:05.520253 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.520229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.523578 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.523543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 18 13:34:05.523706 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.523608    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 18 13:34:05.523706 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.523648    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 18 13:34:05.523823 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.523769    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 18 13:34:05.523823 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.523794    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kjjt4\""
Jul 18 13:34:05.529903 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.529881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"]
Jul 18 13:34:05.549179 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.549157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9m5l\" (UniqueName: \"kubernetes.io/projected/2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f-kube-api-access-c9m5l\") pod \"managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5\" (UID: \"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.549306 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.549209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5\" (UID: \"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.649477 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.649457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5\" (UID: \"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.649583 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.649497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9m5l\" (UniqueName: \"kubernetes.io/projected/2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f-kube-api-access-c9m5l\") pod \"managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5\" (UID: \"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.651869 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.651847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5\" (UID: \"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.657547 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.657525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9m5l\" (UniqueName: \"kubernetes.io/projected/2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f-kube-api-access-c9m5l\") pod \"managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5\" (UID: \"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.840609 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.840583    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"
Jul 18 13:34:05.961540 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:05.961510    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5"]
Jul 18 13:34:05.965003 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:34:05.964974    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ec8b1ef_ad9a_4b3f_b464_9eaabaa2f42f.slice/crio-e03b2ea47faa7427fa7c01d4b2b173777787c193c25a7acb7723b4c52d95ff28 WatchSource:0}: Error finding container e03b2ea47faa7427fa7c01d4b2b173777787c193c25a7acb7723b4c52d95ff28: Status 404 returned error can't find the container with id e03b2ea47faa7427fa7c01d4b2b173777787c193c25a7acb7723b4c52d95ff28
Jul 18 13:34:06.199919 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:06.199858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5" event={"ID":"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f","Type":"ContainerStarted","Data":"e03b2ea47faa7427fa7c01d4b2b173777787c193c25a7acb7723b4c52d95ff28"}
Jul 18 13:34:09.209272 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:09.209229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5" event={"ID":"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f","Type":"ContainerStarted","Data":"1ff9fcb8986dfad41426f9ef25435ef2094ca77c8a94387445468631ac0d7116"}
Jul 18 13:34:09.223394 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:09.223343    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5" podStartSLOduration=1.9218175 podStartE2EDuration="4.223327817s" totalImagesPullingTime="2.301510317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:34:09.222901968 +0000 UTC m=+98.879592100" watchObservedRunningTime="2026-07-18 13:34:09.223327817 +0000 UTC m=+98.880017987"
Jul 18 13:34:10.077762 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:10.077726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:34:10.077909 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:34:10.077872    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:34:10.077964 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:34:10.077940    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:35:14.077920451 +0000 UTC m=+163.734610559 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:34:13.146234 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:13.146196    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xkxjc"
Jul 18 13:34:40.671929 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:34:40.671892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:34:40.672494 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:34:40.672054    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 13:34:40.672494 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:34:40.672143    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs podName:d414579c-495e-4311-9ab8-bc2bae36afd9 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:42.672121043 +0000 UTC m=+252.328811169 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs") pod "network-metrics-daemon-5pfd2" (UID: "d414579c-495e-4311-9ab8-bc2bae36afd9") : secret "metrics-daemon-secret" not found
Jul 18 13:35:09.244504 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:35:09.244461    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9j6vs" podUID="894a67ce-7ec2-4ffb-b31a-3b62e845a2f4"
Jul 18 13:35:09.352427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:35:09.352398    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:35:10.805585 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:35:10.805546    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5pfd2" podUID="d414579c-495e-4311-9ab8-bc2bae36afd9"
Jul 18 13:35:14.083324 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:35:14.083274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:35:14.083769 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:35:14.083412    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 13:35:14.083769 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:35:14.083485    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls podName:894a67ce-7ec2-4ffb-b31a-3b62e845a2f4 nodeName:}" failed. No retries permitted until 2026-07-18 13:37:16.083465147 +0000 UTC m=+285.740155268 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls") pod "dns-default-9j6vs" (UID: "894a67ce-7ec2-4ffb-b31a-3b62e845a2f4") : secret "dns-default-metrics-tls" not found
Jul 18 13:35:23.796679 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:35:23.796583    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:35:58.263193 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:35:58.263156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:08.491998 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:08.491966    2577 generic.go:356] "Generic (PLEG): container finished" podID="2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f" containerID="1ff9fcb8986dfad41426f9ef25435ef2094ca77c8a94387445468631ac0d7116" exitCode=255
Jul 18 13:36:08.492375 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:08.492028    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5" event={"ID":"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f","Type":"ContainerDied","Data":"1ff9fcb8986dfad41426f9ef25435ef2094ca77c8a94387445468631ac0d7116"}
Jul 18 13:36:08.497533 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:08.497514    2577 scope.go:117] "RemoveContainer" containerID="1ff9fcb8986dfad41426f9ef25435ef2094ca77c8a94387445468631ac0d7116"
Jul 18 13:36:09.496140 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:09.496103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bf4ffff94-kg8k5" event={"ID":"2ec8b1ef-ad9a-4b3f-b464-9eaabaa2f42f","Type":"ContainerStarted","Data":"d13df4f13b8363507af6134c606080fa017ee003a28e15c14509a9c9ff815bba"}
Jul 18 13:36:22.394714 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.394678    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"]
Jul 18 13:36:22.397384 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.397368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"
Jul 18 13:36:22.400927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.400907    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.402473 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.402449    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2dp76\""
Jul 18 13:36:22.402947 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.402923    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.403834 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.403720    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2dpwt"]
Jul 18 13:36:22.406503 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.406486    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-766dc69555-trrdr"]
Jul 18 13:36:22.406605 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.406589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.409445 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409418    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.409534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 18 13:36:22.409534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409504    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-5nkpc"]
Jul 18 13:36:22.409642 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.409801 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409780    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.409930 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409780    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 18 13:36:22.410032 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.409958    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dmdbd\""
Jul 18 13:36:22.412306 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.412289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.412980 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.412738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 18 13:36:22.412980 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.412897    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 18 13:36:22.413121 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.413055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xfjwq\""
Jul 18 13:36:22.413471 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.413458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 18 13:36:22.417076 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.417030    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.417397 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.417376    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.420388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.420370    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nw7ds\""
Jul 18 13:36:22.423659 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.423640    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 18 13:36:22.423806 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.423677    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 18 13:36:22.425628 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425598    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 18 13:36:22.425849 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-registry-certificates\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.425940 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-bound-sa-token\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.425940 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sczq\" (UniqueName: \"kubernetes.io/projected/6e46d182-d684-474c-869a-2d4571579100-kube-api-access-6sczq\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.425940 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-trusted-ca\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.426120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-installation-pull-secrets\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.426120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.425976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnxcj\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-kube-api-access-jnxcj\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.426120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.426120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e46d182-d684-474c-869a-2d4571579100-config\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.426120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/261f05d4-c40a-4507-8664-b83ab653895f-ca-trust-extracted\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.426360 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlpf\" (UniqueName: \"kubernetes.io/projected/6aab0b5a-dbc2-4a75-bb02-644272f6dec2-kube-api-access-kqlpf\") pod \"volume-data-source-validator-78cf4c6f6-mb2ch\" (UID: \"6aab0b5a-dbc2-4a75-bb02-644272f6dec2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"
Jul 18 13:36:22.426360 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e46d182-d684-474c-869a-2d4571579100-serving-cert\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.426360 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e46d182-d684-474c-869a-2d4571579100-trusted-ca\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.426360 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-image-registry-private-configuration\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.426835 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.426815    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"]
Jul 18 13:36:22.427489 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.427463    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 18 13:36:22.427584 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.427530    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-5nkpc"]
Jul 18 13:36:22.428610 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.428586    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-766dc69555-trrdr"]
Jul 18 13:36:22.429235 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.429215    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 18 13:36:22.432007 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.431986    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2dpwt"]
Jul 18 13:36:22.487943 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.487919    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"]
Jul 18 13:36:22.490672 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.490658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"
Jul 18 13:36:22.492983 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.492967    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pkksk\""
Jul 18 13:36:22.495284 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.495264    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"]
Jul 18 13:36:22.498169 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.498151    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5b6d57b78-p692k"]
Jul 18 13:36:22.498267 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.498255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.500488 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.500471    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 18 13:36:22.500717 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.500704    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.501008 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.500990    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 18 13:36:22.501094 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.501010    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.501262 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.501246    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"]
Jul 18 13:36:22.501320 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.501305    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qrspl\""
Jul 18 13:36:22.501359 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.501328    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.503494 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.503451    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 18 13:36:22.503625 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.503508    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 18 13:36:22.503791 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.503640    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 18 13:36:22.503791 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.503723    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 13:36:22.503927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.503816    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p6hdz\""
Jul 18 13:36:22.504032 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.504017    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 13:36:22.504111 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.504097    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 18 13:36:22.510883 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.510861    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"]
Jul 18 13:36:22.514211 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.514193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b6d57b78-p692k"]
Jul 18 13:36:22.527382 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jtrw\" (UniqueName: \"kubernetes.io/projected/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-kube-api-access-5jtrw\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.527382 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e57eaa4-6de2-498b-830c-690e2fdce624-tmp\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.527522 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-stats-auth\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.527522 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sczq\" (UniqueName: \"kubernetes.io/projected/6e46d182-d684-474c-869a-2d4571579100-kube-api-access-6sczq\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.527522 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-trusted-ca\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.527648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnxcj\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-kube-api-access-jnxcj\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.527648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/261f05d4-c40a-4507-8664-b83ab653895f-ca-trust-extracted\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.527648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57eaa4-6de2-498b-830c-690e2fdce624-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.527648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5q2m\" (UniqueName: \"kubernetes.io/projected/9e57eaa4-6de2-498b-830c-690e2fdce624-kube-api-access-c5q2m\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.527817 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-image-registry-private-configuration\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.527817 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e57eaa4-6de2-498b-830c-690e2fdce624-snapshots\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.527817 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.527817 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e46d182-d684-474c-869a-2d4571579100-serving-cert\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.527817 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-registry-certificates\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-bound-sa-token\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e46d182-d684-474c-869a-2d4571579100-config\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57eaa4-6de2-498b-830c-690e2fdce624-service-ca-bundle\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-default-certificate\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.527995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-installation-pull-secrets\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528072 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/261f05d4-c40a-4507-8664-b83ab653895f-ca-trust-extracted\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tlq7\" (UniqueName: \"kubernetes.io/projected/d3d2f953-ec65-428d-94f7-222faa182970-kube-api-access-2tlq7\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e46d182-d684-474c-869a-2d4571579100-trusted-ca\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e57eaa4-6de2-498b-830c-690e2fdce624-serving-cert\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:22.528221    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v2sv\" (UniqueName: \"kubernetes.io/projected/c0c94ea7-2a4e-4936-be3f-32a28b26ea56-kube-api-access-2v2sv\") pod \"network-check-source-7bc5b6bf85-kr49n\" (UID: \"c0c94ea7-2a4e-4936-be3f-32a28b26ea56\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlpf\" (UniqueName: \"kubernetes.io/projected/6aab0b5a-dbc2-4a75-bb02-644272f6dec2-kube-api-access-kqlpf\") pod \"volume-data-source-validator-78cf4c6f6-mb2ch\" (UID: \"6aab0b5a-dbc2-4a75-bb02-644272f6dec2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-registry-certificates\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:22.528235    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766dc69555-trrdr: secret "image-registry-tls" not found
Jul 18 13:36:22.528474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.528457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-trusted-ca\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.528876 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:22.528501    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls podName:261f05d4-c40a-4507-8664-b83ab653895f nodeName:}" failed. No retries permitted until 2026-07-18 13:36:23.028482125 +0000 UTC m=+232.685172243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls") pod "image-registry-766dc69555-trrdr" (UID: "261f05d4-c40a-4507-8664-b83ab653895f") : secret "image-registry-tls" not found
Jul 18 13:36:22.529209 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.529188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e46d182-d684-474c-869a-2d4571579100-config\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.529526 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.529510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e46d182-d684-474c-869a-2d4571579100-trusted-ca\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.530256 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.530236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-image-registry-private-configuration\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.530559 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.530541    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e46d182-d684-474c-869a-2d4571579100-serving-cert\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.530627 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.530616    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-installation-pull-secrets\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.540427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.540407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-bound-sa-token\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.540518 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.540483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnxcj\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-kube-api-access-jnxcj\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:22.540608 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.540585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sczq\" (UniqueName: \"kubernetes.io/projected/6e46d182-d684-474c-869a-2d4571579100-kube-api-access-6sczq\") pod \"console-operator-565b7b569d-2dpwt\" (UID: \"6e46d182-d684-474c-869a-2d4571579100\") " pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.540777 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.540762    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlpf\" (UniqueName: \"kubernetes.io/projected/6aab0b5a-dbc2-4a75-bb02-644272f6dec2-kube-api-access-kqlpf\") pod \"volume-data-source-validator-78cf4c6f6-mb2ch\" (UID: \"6aab0b5a-dbc2-4a75-bb02-644272f6dec2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"
Jul 18 13:36:22.619532 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.619498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:22.628630 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.628714 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tlq7\" (UniqueName: \"kubernetes.io/projected/d3d2f953-ec65-428d-94f7-222faa182970-kube-api-access-2tlq7\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.628714 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e57eaa4-6de2-498b-830c-690e2fdce624-serving-cert\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.628714 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v2sv\" (UniqueName: \"kubernetes.io/projected/c0c94ea7-2a4e-4936-be3f-32a28b26ea56-kube-api-access-2v2sv\") pod \"network-check-source-7bc5b6bf85-kr49n\" (UID: \"c0c94ea7-2a4e-4936-be3f-32a28b26ea56\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"
Jul 18 13:36:22.628856 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jtrw\" (UniqueName: \"kubernetes.io/projected/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-kube-api-access-5jtrw\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.628856 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e57eaa4-6de2-498b-830c-690e2fdce624-tmp\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.628856 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-stats-auth\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.628856 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57eaa4-6de2-498b-830c-690e2fdce624-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5q2m\" (UniqueName: \"kubernetes.io/projected/9e57eaa4-6de2-498b-830c-690e2fdce624-kube-api-access-c5q2m\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e57eaa4-6de2-498b-830c-690e2fdce624-snapshots\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.628957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57eaa4-6de2-498b-830c-690e2fdce624-service-ca-bundle\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-default-certificate\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.629075 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.629434 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e57eaa4-6de2-498b-830c-690e2fdce624-tmp\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629585 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.629655 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57eaa4-6de2-498b-830c-690e2fdce624-service-ca-bundle\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629655 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:22.629650    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:23.129634246 +0000 UTC m=+232.786324354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : configmap references non-existent config key: service-ca.crt
Jul 18 13:36:22.629777 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:22.629709    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 13:36:22.629777 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629749    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e57eaa4-6de2-498b-830c-690e2fdce624-snapshots\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.629777 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:22.629765    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:23.129748315 +0000 UTC m=+232.786438425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : secret "router-metrics-certs-default" not found
Jul 18 13:36:22.629994 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.629969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e57eaa4-6de2-498b-830c-690e2fdce624-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.631103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.631083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.631216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.631199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e57eaa4-6de2-498b-830c-690e2fdce624-serving-cert\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.631552 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.631531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-default-certificate\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.631725 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.631709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-stats-auth\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.639404 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.639382    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5q2m\" (UniqueName: \"kubernetes.io/projected/9e57eaa4-6de2-498b-830c-690e2fdce624-kube-api-access-c5q2m\") pod \"insights-operator-555d9d7885-5nkpc\" (UID: \"9e57eaa4-6de2-498b-830c-690e2fdce624\") " pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.639504 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.639470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tlq7\" (UniqueName: \"kubernetes.io/projected/d3d2f953-ec65-428d-94f7-222faa182970-kube-api-access-2tlq7\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:22.639504 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.639493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jtrw\" (UniqueName: \"kubernetes.io/projected/b59163b6-7d4a-4c22-ae0c-98ffaefc7f35-kube-api-access-5jtrw\") pod \"kube-storage-version-migrator-operator-7658d9b749-jmc6r\" (UID: \"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.639859 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.639844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v2sv\" (UniqueName: \"kubernetes.io/projected/c0c94ea7-2a4e-4936-be3f-32a28b26ea56-kube-api-access-2v2sv\") pod \"network-check-source-7bc5b6bf85-kr49n\" (UID: \"c0c94ea7-2a4e-4936-be3f-32a28b26ea56\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"
Jul 18 13:36:22.706764 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.706714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"
Jul 18 13:36:22.717115 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.717099    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:22.723414 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.723388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:22.735363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.735337    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-5nkpc"
Jul 18 13:36:22.799334 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.799251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"
Jul 18 13:36:22.808533 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.808330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"
Jul 18 13:36:22.861008 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.860926    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch"]
Jul 18 13:36:22.864844 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:22.864797    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aab0b5a_dbc2_4a75_bb02_644272f6dec2.slice/crio-2a3d1513c60cfe5eed3bbb93e66b8dace70e6522eed98177ea7c0afddbffd994 WatchSource:0}: Error finding container 2a3d1513c60cfe5eed3bbb93e66b8dace70e6522eed98177ea7c0afddbffd994: Status 404 returned error can't find the container with id 2a3d1513c60cfe5eed3bbb93e66b8dace70e6522eed98177ea7c0afddbffd994
Jul 18 13:36:22.890022 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.889991    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2dpwt"]
Jul 18 13:36:22.903703 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.903676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-5nkpc"]
Jul 18 13:36:22.907570 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:22.907536    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e57eaa4_6de2_498b_830c_690e2fdce624.slice/crio-2f42a3eef60e02868f68cf293284506f50cd8d495643aeb67a64066905b83b0f WatchSource:0}: Error finding container 2f42a3eef60e02868f68cf293284506f50cd8d495643aeb67a64066905b83b0f: Status 404 returned error can't find the container with id 2f42a3eef60e02868f68cf293284506f50cd8d495643aeb67a64066905b83b0f
Jul 18 13:36:22.953848 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.953814    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n"]
Jul 18 13:36:22.956698 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:22.956671    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c94ea7_2a4e_4936_be3f_32a28b26ea56.slice/crio-aa9295afd7c852dcbd7b208e2e1279929b625ac4524573e483ec728d4765f6c5 WatchSource:0}: Error finding container aa9295afd7c852dcbd7b208e2e1279929b625ac4524573e483ec728d4765f6c5: Status 404 returned error can't find the container with id aa9295afd7c852dcbd7b208e2e1279929b625ac4524573e483ec728d4765f6c5
Jul 18 13:36:22.970812 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:22.970794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r"]
Jul 18 13:36:22.973925 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:22.973903    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59163b6_7d4a_4c22_ae0c_98ffaefc7f35.slice/crio-50e3efd2f6a9cdb334b9149f12ce0f1a792e9075ebfafe43651792380c3c1261 WatchSource:0}: Error finding container 50e3efd2f6a9cdb334b9149f12ce0f1a792e9075ebfafe43651792380c3c1261: Status 404 returned error can't find the container with id 50e3efd2f6a9cdb334b9149f12ce0f1a792e9075ebfafe43651792380c3c1261
Jul 18 13:36:23.033925 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.033894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:23.034085 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:23.034013    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:36:23.034085 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:23.034032    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766dc69555-trrdr: secret "image-registry-tls" not found
Jul 18 13:36:23.034170 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:23.034094    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls podName:261f05d4-c40a-4507-8664-b83ab653895f nodeName:}" failed. No retries permitted until 2026-07-18 13:36:24.034078931 +0000 UTC m=+233.690769041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls") pod "image-registry-766dc69555-trrdr" (UID: "261f05d4-c40a-4507-8664-b83ab653895f") : secret "image-registry-tls" not found
Jul 18 13:36:23.134553 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.134527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:23.134700 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.134565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:23.134700 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:23.134663    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 13:36:23.134808 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:23.134699    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:24.134679155 +0000 UTC m=+233.791369293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : configmap references non-existent config key: service-ca.crt
Jul 18 13:36:23.134808 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:23.134727    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:24.134717342 +0000 UTC m=+233.791407449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : secret "router-metrics-certs-default" not found
Jul 18 13:36:23.545168 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.545070    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r" event={"ID":"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35","Type":"ContainerStarted","Data":"50e3efd2f6a9cdb334b9149f12ce0f1a792e9075ebfafe43651792380c3c1261"}
Jul 18 13:36:23.549021 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.548905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n" event={"ID":"c0c94ea7-2a4e-4936-be3f-32a28b26ea56","Type":"ContainerStarted","Data":"a19c4e120b029b17c84258ae3ae089e73f93c94c89d850832afefdfa14819510"}
Jul 18 13:36:23.549021 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.548947    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n" event={"ID":"c0c94ea7-2a4e-4936-be3f-32a28b26ea56","Type":"ContainerStarted","Data":"aa9295afd7c852dcbd7b208e2e1279929b625ac4524573e483ec728d4765f6c5"}
Jul 18 13:36:23.552577 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.552492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-5nkpc" event={"ID":"9e57eaa4-6de2-498b-830c-690e2fdce624","Type":"ContainerStarted","Data":"2f42a3eef60e02868f68cf293284506f50cd8d495643aeb67a64066905b83b0f"}
Jul 18 13:36:23.554943 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.554882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" event={"ID":"6e46d182-d684-474c-869a-2d4571579100","Type":"ContainerStarted","Data":"357403eed874af5406c32c164397d76e2e7e4b66003de6caad8174812db8ad31"}
Jul 18 13:36:23.556739 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:23.556688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch" event={"ID":"6aab0b5a-dbc2-4a75-bb02-644272f6dec2","Type":"ContainerStarted","Data":"2a3d1513c60cfe5eed3bbb93e66b8dace70e6522eed98177ea7c0afddbffd994"}
Jul 18 13:36:24.042366 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.042332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:24.042568 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:24.042547    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:36:24.042568 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:24.042568    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766dc69555-trrdr: secret "image-registry-tls" not found
Jul 18 13:36:24.042694 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:24.042626    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls podName:261f05d4-c40a-4507-8664-b83ab653895f nodeName:}" failed. No retries permitted until 2026-07-18 13:36:26.042606459 +0000 UTC m=+235.699296572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls") pod "image-registry-766dc69555-trrdr" (UID: "261f05d4-c40a-4507-8664-b83ab653895f") : secret "image-registry-tls" not found
Jul 18 13:36:24.143889 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.143847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:24.144092 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.143921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:24.144164 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:24.144132    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 13:36:24.144211 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:24.144195    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:26.144176364 +0000 UTC m=+235.800866472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : secret "router-metrics-certs-default" not found
Jul 18 13:36:24.144646 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:24.144606    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:26.144587851 +0000 UTC m=+235.801277961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : configmap references non-existent config key: service-ca.crt
Jul 18 13:36:24.562217 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.562146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch" event={"ID":"6aab0b5a-dbc2-4a75-bb02-644272f6dec2","Type":"ContainerStarted","Data":"6437d3222d18468fdf3d743e9f3fbfee9829134c941d947a999fdee6bd053e9c"}
Jul 18 13:36:24.577505 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.577322    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kr49n" podStartSLOduration=2.577303346 podStartE2EDuration="2.577303346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:23.563918577 +0000 UTC m=+233.220608719" watchObservedRunningTime="2026-07-18 13:36:24.577303346 +0000 UTC m=+234.233993475"
Jul 18 13:36:24.577994 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.577856    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-mb2ch" podStartSLOduration=1.262170336 podStartE2EDuration="2.577843365s" totalImagesPullingTime="1.315673029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:24.577150497 +0000 UTC m=+234.233840628" watchObservedRunningTime="2026-07-18 13:36:24.577843365 +0000 UTC m=+234.234533495"
Jul 18 13:36:24.651401 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.651370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:24.703992 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:24.703958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:25.736462 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:25.736418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:25.789025 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:25.788987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.061841 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.061819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:26.061942 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:26.061933    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:36:26.061988 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:26.061944    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766dc69555-trrdr: secret "image-registry-tls" not found
Jul 18 13:36:26.061988 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:26.061986    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls podName:261f05d4-c40a-4507-8664-b83ab653895f nodeName:}" failed. No retries permitted until 2026-07-18 13:36:30.061973936 +0000 UTC m=+239.718664043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls") pod "image-registry-766dc69555-trrdr" (UID: "261f05d4-c40a-4507-8664-b83ab653895f") : secret "image-registry-tls" not found
Jul 18 13:36:26.162191 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.162166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:26.162320 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.162206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:26.162385 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:26.162334    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 13:36:26.162385 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:26.162364    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:30.162340511 +0000 UTC m=+239.819030632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : configmap references non-existent config key: service-ca.crt
Jul 18 13:36:26.162501 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:26.162391    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:30.162379985 +0000 UTC m=+239.819070092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : secret "router-metrics-certs-default" not found
Jul 18 13:36:26.173375 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.173334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.277544 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.274274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.442686 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.442611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.543559 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.543532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.573106 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.573075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r" event={"ID":"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35","Type":"ContainerStarted","Data":"2f7fa29030c655a2f08478901a2d10642ff9a59dd7d076652146469b9225e4e9"}
Jul 18 13:36:26.574561 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.574529    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-5nkpc" event={"ID":"9e57eaa4-6de2-498b-830c-690e2fdce624","Type":"ContainerStarted","Data":"e088acf6dcfb78ef23984d1db2f3159b2c4a07a4fa8920000f4359f76dab57d9"}
Jul 18 13:36:26.576153 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.576134    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/0.log"
Jul 18 13:36:26.576235 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.576171    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e46d182-d684-474c-869a-2d4571579100" containerID="551fae94b34021645512aa1d4a0932353293bb890f2c91a4b375dfd20f9e6e62" exitCode=255
Jul 18 13:36:26.576235 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.576198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" event={"ID":"6e46d182-d684-474c-869a-2d4571579100","Type":"ContainerDied","Data":"551fae94b34021645512aa1d4a0932353293bb890f2c91a4b375dfd20f9e6e62"}
Jul 18 13:36:26.576452 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.576434    2577 scope.go:117] "RemoveContainer" containerID="551fae94b34021645512aa1d4a0932353293bb890f2c91a4b375dfd20f9e6e62"
Jul 18 13:36:26.590387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.590343    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r" podStartSLOduration=1.506611249 podStartE2EDuration="4.590325374s" totalImagesPullingTime="3.083714125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:26.589921466 +0000 UTC m=+236.246611596" watchObservedRunningTime="2026-07-18 13:36:26.590325374 +0000 UTC m=+236.247015498"
Jul 18 13:36:26.608977 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.607979    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-5nkpc" podStartSLOduration=1.4449603739999999 podStartE2EDuration="4.607964381s" totalImagesPullingTime="3.163004007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:26.606841108 +0000 UTC m=+236.263531238" watchObservedRunningTime="2026-07-18 13:36:26.607964381 +0000 UTC m=+236.264654512"
Jul 18 13:36:26.771649 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.771610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.871298 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.871269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:26.926388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:26.926360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:27.580976 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.580947    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:36:27.581387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.581371    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/0.log"
Jul 18 13:36:27.581488 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.581404    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e46d182-d684-474c-869a-2d4571579100" containerID="b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393" exitCode=255
Jul 18 13:36:27.581488 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.581434    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" event={"ID":"6e46d182-d684-474c-869a-2d4571579100","Type":"ContainerDied","Data":"b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393"}
Jul 18 13:36:27.581557 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.581488    2577 scope.go:117] "RemoveContainer" containerID="551fae94b34021645512aa1d4a0932353293bb890f2c91a4b375dfd20f9e6e62"
Jul 18 13:36:27.581721 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.581701    2577 scope.go:117] "RemoveContainer" containerID="b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393"
Jul 18 13:36:27.581914 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:27.581895    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2dpwt_openshift-console-operator(6e46d182-d684-474c-869a-2d4571579100)\"" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" podUID="6e46d182-d684-474c-869a-2d4571579100"
Jul 18 13:36:27.805487 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.805453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:27.862632 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:27.862568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:28.057583 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.057553    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-dqzl2"]
Jul 18 13:36:28.061642 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.061618    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2hlkh"]
Jul 18 13:36:28.061780 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.061735    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.064024 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.064004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 18 13:36:28.064135 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.064089    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 18 13:36:28.064135 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.064127    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 18 13:36:28.064222 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.064158    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cwpvk\""
Jul 18 13:36:28.064424 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.064409    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 18 13:36:28.064946 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.064926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.067186 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.067168    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 13:36:28.067303 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.067206    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8pprp\""
Jul 18 13:36:28.067303 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.067263    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 13:36:28.071549 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.071529    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-dqzl2"]
Jul 18 13:36:28.072569 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.072550    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2hlkh"]
Jul 18 13:36:28.093774 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.093738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:28.150306 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.150242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:28.176413 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vnns\" (UniqueName: \"kubernetes.io/projected/02bb10cf-2766-4287-bf46-d54cdaf85320-kube-api-access-8vnns\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.176509 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/02bb10cf-2766-4287-bf46-d54cdaf85320-signing-cabundle\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.176509 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/02bb10cf-2766-4287-bf46-d54cdaf85320-signing-key\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.176590 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b4d5a23-b14b-4260-87d0-0f838919b077-data-volume\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.176590 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.176590 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b4d5a23-b14b-4260-87d0-0f838919b077-crio-socket\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.176727 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.176727 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.176659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t429w\" (UniqueName: \"kubernetes.io/projected/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-api-access-t429w\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.277872 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.277848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vnns\" (UniqueName: \"kubernetes.io/projected/02bb10cf-2766-4287-bf46-d54cdaf85320-kube-api-access-8vnns\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.278008 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.277886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/02bb10cf-2766-4287-bf46-d54cdaf85320-signing-cabundle\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.278008 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.277903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/02bb10cf-2766-4287-bf46-d54cdaf85320-signing-key\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.278008 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.277922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b4d5a23-b14b-4260-87d0-0f838919b077-data-volume\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278218 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278218 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b4d5a23-b14b-4260-87d0-0f838919b077-crio-socket\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278218 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278369 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t429w\" (UniqueName: \"kubernetes.io/projected/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-api-access-t429w\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278369 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.278241    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.278369 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b4d5a23-b14b-4260-87d0-0f838919b077-crio-socket\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278369 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b4d5a23-b14b-4260-87d0-0f838919b077-data-volume\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.278369 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.278307    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:28.778285814 +0000 UTC m=+238.434975935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.278369 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.278310    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.278582 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.278382    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:28.778364456 +0000 UTC m=+238.435054584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.278635 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.278616    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/02bb10cf-2766-4287-bf46-d54cdaf85320-signing-cabundle\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.280459 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.280438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/02bb10cf-2766-4287-bf46-d54cdaf85320-signing-key\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.286154 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.286130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t429w\" (UniqueName: \"kubernetes.io/projected/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-api-access-t429w\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.286228 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.286205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vnns\" (UniqueName: \"kubernetes.io/projected/02bb10cf-2766-4287-bf46-d54cdaf85320-kube-api-access-8vnns\") pod \"service-ca-58598695f7-dqzl2\" (UID: \"02bb10cf-2766-4287-bf46-d54cdaf85320\") " pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.372588 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.372572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-dqzl2"
Jul 18 13:36:28.493163 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.493141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-dqzl2"]
Jul 18 13:36:28.495380 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:28.495355    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bb10cf_2766_4287_bf46_d54cdaf85320.slice/crio-74ac0456550572fee5a3a9861f3cb0b0000f664214f6a61f14ac246c7d79bc00 WatchSource:0}: Error finding container 74ac0456550572fee5a3a9861f3cb0b0000f664214f6a61f14ac246c7d79bc00: Status 404 returned error can't find the container with id 74ac0456550572fee5a3a9861f3cb0b0000f664214f6a61f14ac246c7d79bc00
Jul 18 13:36:28.584861 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.584836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-dqzl2" event={"ID":"02bb10cf-2766-4287-bf46-d54cdaf85320","Type":"ContainerStarted","Data":"74ac0456550572fee5a3a9861f3cb0b0000f664214f6a61f14ac246c7d79bc00"}
Jul 18 13:36:28.586112 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.586095    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:36:28.586388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.586374    2577 scope.go:117] "RemoveContainer" containerID="b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393"
Jul 18 13:36:28.586517 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.586501    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2dpwt_openshift-console-operator(6e46d182-d684-474c-869a-2d4571579100)\"" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" podUID="6e46d182-d684-474c-869a-2d4571579100"
Jul 18 13:36:28.782779 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.782755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.782878 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.782797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:28.782878 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.782851    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.782950 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.782882    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.782950 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.782894    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:29.782881377 +0000 UTC m=+239.439571484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:28.782950 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:28.782924    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:29.782912038 +0000 UTC m=+239.439602145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:28.913260 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:28.913221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:29.791006 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:29.790966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:29.791195 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:29.791024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:29.791195 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:29.791122    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:29.791195 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:29.791163    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:29.791195 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:29.791180    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:31.791166011 +0000 UTC m=+241.447856118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:29.791370 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:29.791244    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:31.791224967 +0000 UTC m=+241.447915080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:30.094415 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:30.094324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:30.094839 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:30.094482    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 13:36:30.094839 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:30.094499    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766dc69555-trrdr: secret "image-registry-tls" not found
Jul 18 13:36:30.094839 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:30.094564    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls podName:261f05d4-c40a-4507-8664-b83ab653895f nodeName:}" failed. No retries permitted until 2026-07-18 13:36:38.094544527 +0000 UTC m=+247.751234642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls") pod "image-registry-766dc69555-trrdr" (UID: "261f05d4-c40a-4507-8664-b83ab653895f") : secret "image-registry-tls" not found
Jul 18 13:36:30.195464 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:30.195432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:30.195613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:30.195527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:30.195613 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:30.195572    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 13:36:30.195679 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:30.195635    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:38.195620422 +0000 UTC m=+247.852310532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : secret "router-metrics-certs-default" not found
Jul 18 13:36:30.195679 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:30.195650    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle podName:d3d2f953-ec65-428d-94f7-222faa182970 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:38.195642542 +0000 UTC m=+247.852332650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle") pod "router-default-5b6d57b78-p692k" (UID: "d3d2f953-ec65-428d-94f7-222faa182970") : configmap references non-existent config key: service-ca.crt
Jul 18 13:36:30.595996 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:30.595964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-dqzl2" event={"ID":"02bb10cf-2766-4287-bf46-d54cdaf85320","Type":"ContainerStarted","Data":"9cd70be867df42c1428e0b0f5bd72bb155feccca7dec991b1118b1263e9119e8"}
Jul 18 13:36:30.611268 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:30.611224    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-dqzl2" podStartSLOduration=0.877967705 podStartE2EDuration="2.611209116s" totalImagesPullingTime="1.733241411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:30.609956674 +0000 UTC m=+240.266646803" watchObservedRunningTime="2026-07-18 13:36:30.611209116 +0000 UTC m=+240.267899244"
Jul 18 13:36:30.839477 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:30.839447    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9fnb7_9dd4addb-d362-4a5a-96ff-4bc28f94e2f3/dns-node-resolver/0.log"
Jul 18 13:36:31.347918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:31.347878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:31.640207 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:31.640131    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4v2rl_cb9103a2-100d-4187-91fc-8708bbf5cd35/node-ca/0.log"
Jul 18 13:36:31.807855 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:31.807801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:31.808004 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:31.807862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:31.808004 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:31.807950    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:31.808004 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:31.807965    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:31.808230 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:31.808021    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:35.808000096 +0000 UTC m=+245.464690209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:31.808230 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:31.808052    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:35.808032126 +0000 UTC m=+245.464722233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:32.368462 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:32.368430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:32.639839 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:32.639766    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7hw6q_19c69f7e-d198-4022-abaf-cb8b8f3b329b/serve-healthcheck-canary/0.log"
Jul 18 13:36:32.717927 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:32.717896    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:32.718101 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:32.717965    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:32.718313 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:32.718297    2577 scope.go:117] "RemoveContainer" containerID="b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393"
Jul 18 13:36:32.718482 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:32.718460    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2dpwt_openshift-console-operator(6e46d182-d684-474c-869a-2d4571579100)\"" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" podUID="6e46d182-d684-474c-869a-2d4571579100"
Jul 18 13:36:33.028723 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:33.028680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:33.435407 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:33.435309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:33.440719 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:33.440694    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-jmc6r_b59163b6-7d4a-4c22-ae0c-98ffaefc7f35/kube-storage-version-migrator-operator/0.log"
Jul 18 13:36:33.607846 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:33.607825    2577 scope.go:117] "RemoveContainer" containerID="b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393"
Jul 18 13:36:33.607988 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:33.607970    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2dpwt_openshift-console-operator(6e46d182-d684-474c-869a-2d4571579100)\"" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" podUID="6e46d182-d684-474c-869a-2d4571579100"
Jul 18 13:36:33.632317 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:33.632288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:33.829515 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:33.829486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:34.234658 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:34.234577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:34.832259 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:34.832231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:34.972982 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:34.972949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:35.229829 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:35.229738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:35.281311 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:35.281281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:35.632383 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:35.632350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:35.838386 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:35.838358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:35.838700 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:35.838413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:35.838700 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:35.838458    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:35.838700 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:35.838477    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 13:36:35.838700 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:35.838513    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:43.838499167 +0000 UTC m=+253.495189274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : secret "insights-runtime-extractor-tls" not found
Jul 18 13:36:35.838700 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:36:35.838528    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm podName:1b4d5a23-b14b-4260-87d0-0f838919b077 nodeName:}" failed. No retries permitted until 2026-07-18 13:36:43.838521484 +0000 UTC m=+253.495211591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hlkh" (UID: "1b4d5a23-b14b-4260-87d0-0f838919b077") : configmap "kube-rbac-proxy" not found
Jul 18 13:36:36.029619 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:36.029591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:36.230347 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:36.230297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:36.831380 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:36.831343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:37.231596 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:37.231517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:37.632837 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:37.632795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:37.823458 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:37.823434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:37.872148 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:37.872121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:38.155873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.155848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:38.158159 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.158140    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"image-registry-766dc69555-trrdr\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") " pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:38.232866 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.232835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:38.256832 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.256806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:38.256941 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.256849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:38.257374 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.257357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d2f953-ec65-428d-94f7-222faa182970-service-ca-bundle\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:38.259282 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.259260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d2f953-ec65-428d-94f7-222faa182970-metrics-certs\") pod \"router-default-5b6d57b78-p692k\" (UID: \"d3d2f953-ec65-428d-94f7-222faa182970\") " pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:38.332136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.332111    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xfjwq\""
Jul 18 13:36:38.340921 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.340903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:38.416622 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.416561    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p6hdz\""
Jul 18 13:36:38.425319 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.425293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:38.466929 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.466906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-766dc69555-trrdr"]
Jul 18 13:36:38.469209 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:38.469181    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261f05d4_c40a_4507_8664_b83ab653895f.slice/crio-20eeb5a339faeda47483e80ebb547f8627e391d8978c4a70adda211ed43c4e38 WatchSource:0}: Error finding container 20eeb5a339faeda47483e80ebb547f8627e391d8978c4a70adda211ed43c4e38: Status 404 returned error can't find the container with id 20eeb5a339faeda47483e80ebb547f8627e391d8978c4a70adda211ed43c4e38
Jul 18 13:36:38.549669 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.549629    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b6d57b78-p692k"]
Jul 18 13:36:38.551549 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:38.551528    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d2f953_ec65_428d_94f7_222faa182970.slice/crio-aafe0e7f80e4984336b22257875f03a7e97023f05fff9b6dcaeb361ad3237b3d WatchSource:0}: Error finding container aafe0e7f80e4984336b22257875f03a7e97023f05fff9b6dcaeb361ad3237b3d: Status 404 returned error can't find the container with id aafe0e7f80e4984336b22257875f03a7e97023f05fff9b6dcaeb361ad3237b3d
Jul 18 13:36:38.630121 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.630084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766dc69555-trrdr" event={"ID":"261f05d4-c40a-4507-8664-b83ab653895f","Type":"ContainerStarted","Data":"5c553339584dd6c76e64cf1cd18e5f9f361f4d4c0d7ff61b108f597b6e4ced0e"}
Jul 18 13:36:38.630257 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.630218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:36:38.630328 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.630258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766dc69555-trrdr" event={"ID":"261f05d4-c40a-4507-8664-b83ab653895f","Type":"ContainerStarted","Data":"20eeb5a339faeda47483e80ebb547f8627e391d8978c4a70adda211ed43c4e38"}
Jul 18 13:36:38.631780 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.631757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b6d57b78-p692k" event={"ID":"d3d2f953-ec65-428d-94f7-222faa182970","Type":"ContainerStarted","Data":"f9dd8b8e6c5a70aa9b86674e6fd70f935f223a50c79f711412cab3c8911e62b0"}
Jul 18 13:36:38.631888 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.631797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b6d57b78-p692k" event={"ID":"d3d2f953-ec65-428d-94f7-222faa182970","Type":"ContainerStarted","Data":"aafe0e7f80e4984336b22257875f03a7e97023f05fff9b6dcaeb361ad3237b3d"}
Jul 18 13:36:38.646783 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.646725    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-766dc69555-trrdr" podStartSLOduration=16.646707212 podStartE2EDuration="16.646707212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:38.645822844 +0000 UTC m=+248.302512964" watchObservedRunningTime="2026-07-18 13:36:38.646707212 +0000 UTC m=+248.303397343"
Jul 18 13:36:38.663217 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.663170    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5b6d57b78-p692k" podStartSLOduration=16.663157348 podStartE2EDuration="16.663157348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:38.661868702 +0000 UTC m=+248.318558838" watchObservedRunningTime="2026-07-18 13:36:38.663157348 +0000 UTC m=+248.319847476"
Jul 18 13:36:38.833952 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:38.833921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:39.378893 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.378854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:39.425739 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.425710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:39.428241 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.428221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:39.453092 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.453067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:39.631015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.630943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:39.635536 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.635517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:39.636610 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.636593    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5b6d57b78-p692k"
Jul 18 13:36:39.831627 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:39.831600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:40.035031 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:40.035002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:41.234179 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:41.234144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:41.432576 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:41.432547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:41.632060 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:41.632014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:41.829000 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:41.828962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:42.037174 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:42.037144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:42.691301 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:42.691260    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:36:42.693655 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:42.693628    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d414579c-495e-4311-9ab8-bc2bae36afd9-metrics-certs\") pod \"network-metrics-daemon-5pfd2\" (UID: \"d414579c-495e-4311-9ab8-bc2bae36afd9\") " pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:36:42.699767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:42.699748    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l5j6b\""
Jul 18 13:36:42.708457 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:42.708441    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5pfd2"
Jul 18 13:36:42.851915 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:42.851893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5pfd2"]
Jul 18 13:36:42.854175 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:42.854152    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd414579c_495e_4311_9ab8_bc2bae36afd9.slice/crio-58ca758871af8762aa21e02e35235b93323568e767f5f2b957008c8eeeae909e WatchSource:0}: Error finding container 58ca758871af8762aa21e02e35235b93323568e767f5f2b957008c8eeeae909e: Status 404 returned error can't find the container with id 58ca758871af8762aa21e02e35235b93323568e767f5f2b957008c8eeeae909e
Jul 18 13:36:43.651077 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.651020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5pfd2" event={"ID":"d414579c-495e-4311-9ab8-bc2bae36afd9","Type":"ContainerStarted","Data":"58ca758871af8762aa21e02e35235b93323568e767f5f2b957008c8eeeae909e"}
Jul 18 13:36:43.915738 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.915708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:43.916012 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.915751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:43.916268 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.916244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b4d5a23-b14b-4260-87d0-0f838919b077-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:43.917768 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.917744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b4d5a23-b14b-4260-87d0-0f838919b077-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hlkh\" (UID: \"1b4d5a23-b14b-4260-87d0-0f838919b077\") " pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:43.979911 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.979886    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8pprp\""
Jul 18 13:36:43.988695 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:43.988678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2hlkh"
Jul 18 13:36:44.111570 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:44.111547    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2hlkh"]
Jul 18 13:36:44.113544 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:36:44.113519    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4d5a23_b14b_4260_87d0_0f838919b077.slice/crio-9c52037668ae04924f2c1836ce702040d76ad01ebe551f6f1f842e008829824b WatchSource:0}: Error finding container 9c52037668ae04924f2c1836ce702040d76ad01ebe551f6f1f842e008829824b: Status 404 returned error can't find the container with id 9c52037668ae04924f2c1836ce702040d76ad01ebe551f6f1f842e008829824b
Jul 18 13:36:44.658661 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:44.658622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5pfd2" event={"ID":"d414579c-495e-4311-9ab8-bc2bae36afd9","Type":"ContainerStarted","Data":"9f578a22a90b6641919710315a4253ef5f3fb9268db0a265d166a9219b8f7750"}
Jul 18 13:36:44.658838 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:44.658693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5pfd2" event={"ID":"d414579c-495e-4311-9ab8-bc2bae36afd9","Type":"ContainerStarted","Data":"3f7d11b1aed5cece39b470fb6c82e00dbf106d783d12c2c28d12c00c7f11d277"}
Jul 18 13:36:44.660125 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:44.660089    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hlkh" event={"ID":"1b4d5a23-b14b-4260-87d0-0f838919b077","Type":"ContainerStarted","Data":"9c52037668ae04924f2c1836ce702040d76ad01ebe551f6f1f842e008829824b"}
Jul 18 13:36:44.675340 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:44.675290    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5pfd2" podStartSLOduration=252.791466284 podStartE2EDuration="4m13.675273836s" totalImagesPullingTime="883.807552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:44.672825106 +0000 UTC m=+254.329515267" watchObservedRunningTime="2026-07-18 13:36:44.675273836 +0000 UTC m=+254.331963964"
Jul 18 13:36:46.796478 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:46.796440    2577 scope.go:117] "RemoveContainer" containerID="b3fa87a6f6b2182d3641e23558e943bb0a33d5a9bfe0ce7bf87729ba245f3393"
Jul 18 13:36:48.242024 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:48.241942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:48.255263 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:48.255234    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-766dc69555-trrdr"]
Jul 18 13:36:48.373082 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:48.373018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:48.468904 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:48.468771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:49.481436 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.481348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:49.580150 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.580115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:49.633656 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.633567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:49.681886 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.681789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hlkh" event={"ID":"1b4d5a23-b14b-4260-87d0-0f838919b077","Type":"ContainerStarted","Data":"7172df697884f61418b18e518031b0b5cbf6f5711ccf863ce5dc1069d51cff88"}
Jul 18 13:36:49.684155 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.684131    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:36:49.684285 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.684196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" event={"ID":"6e46d182-d684-474c-869a-2d4571579100","Type":"ContainerStarted","Data":"9f90561fea41f6076ab68fbda3514ee56176effa73d5eafa65c574ae0b0033da"}
Jul 18 13:36:49.684779 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.684734    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:49.691058 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.690998    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt"
Jul 18 13:36:49.701738 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.701686    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-2dpwt" podStartSLOduration=24.522376818 podStartE2EDuration="27.701667529s" totalImagesPullingTime="3.179290711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:49.699497183 +0000 UTC m=+259.356187314" watchObservedRunningTime="2026-07-18 13:36:49.701667529 +0000 UTC m=+259.358357659"
Jul 18 13:36:49.767187 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:49.767031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.145327 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.145289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.197341 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.197307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.353395 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.353359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.542227 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.542185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.594504 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.594448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.878261 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.878147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:50.977162 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:50.977122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:51.030107 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:51.030054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:51.162985 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:51.162894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:51.260576 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:51.260536    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:52.570608 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:52.570584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:52.672208 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:52.672141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:52.696581 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:52.696557    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hlkh" event={"ID":"1b4d5a23-b14b-4260-87d0-0f838919b077","Type":"ContainerStarted","Data":"5b91b629280efc068c6e85962fa24790cdda6564651a8eeb063f82cad0004be2"}
Jul 18 13:36:56.710982 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:56.710941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hlkh" event={"ID":"1b4d5a23-b14b-4260-87d0-0f838919b077","Type":"ContainerStarted","Data":"8882e3aa4c5803195bc28067d1c4a1ca116339f755ff9520ec8a8f22217e9f1a"}
Jul 18 13:36:56.727518 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:56.727465    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2hlkh" podStartSLOduration=17.250151192 podStartE2EDuration="28.727450875s" totalImagesPullingTime="11.477299683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 13:36:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:36:56.726269206 +0000 UTC m=+266.382959336" watchObservedRunningTime="2026-07-18 13:36:56.727450875 +0000 UTC m=+266.384141004"
Jul 18 13:36:58.261186 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:58.261147    2577 patch_prober.go:28] interesting pod/image-registry-766dc69555-trrdr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 18 13:36:58.261601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:58.261204    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-766dc69555-trrdr" podUID="261f05d4-c40a-4507-8664-b83ab653895f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 13:36:58.389189 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:58.389155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:58.445566 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:58.445534    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:58.604309 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:58.604277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:58.758030 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:58.757979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:36:59.012300 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:36:59.012218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:00.968164 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:00.968128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:01.066737 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:01.066705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:02.581408 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.581374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:02.633787 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.633754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:02.788750 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.788718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:02.821203 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.821182    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"]
Jul 18 13:37:02.824218 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.824203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.827259 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.827237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 13:37:02.827365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.827244    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 18 13:37:02.827365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.827272    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 18 13:37:02.827365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.827264    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 13:37:02.827365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.827342    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 13:37:02.827625 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.827596    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qm7rv\""
Jul 18 13:37:02.831828 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.831781    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"]
Jul 18 13:37:02.851833 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.851794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:02.862918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.862893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.863014 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.862941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.863014 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.862961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbj2\" (UniqueName: \"kubernetes.io/projected/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-kube-api-access-5fbj2\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.863123 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.863018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.963661 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.963638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.963757 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.963694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.963757 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.963732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.963757 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.963750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbj2\" (UniqueName: \"kubernetes.io/projected/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-kube-api-access-5fbj2\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.964358 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.964341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.966103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.966088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.966168 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.966108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.970674 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.970646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbj2\" (UniqueName: \"kubernetes.io/projected/a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df-kube-api-access-5fbj2\") pod \"prometheus-operator-5945b5cfd-bfvjq\" (UID: \"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:02.983558 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:02.983535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:03.133783 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:03.133729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"
Jul 18 13:37:03.261710 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:03.261671    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq"]
Jul 18 13:37:03.267237 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:03.267211    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ac7f31_4b48_44a3_a5e6_d2ba32cee1df.slice/crio-8e8841d8a4bd30a1a509a0ccad745920d66ae9ef0757c3fb30d8d8e2ef024a45 WatchSource:0}: Error finding container 8e8841d8a4bd30a1a509a0ccad745920d66ae9ef0757c3fb30d8d8e2ef024a45: Status 404 returned error can't find the container with id 8e8841d8a4bd30a1a509a0ccad745920d66ae9ef0757c3fb30d8d8e2ef024a45
Jul 18 13:37:03.695061 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:03.695005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:03.738381 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:03.738352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq" event={"ID":"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df","Type":"ContainerStarted","Data":"8e8841d8a4bd30a1a509a0ccad745920d66ae9ef0757c3fb30d8d8e2ef024a45"}
Jul 18 13:37:04.382771 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:04.382732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:04.699198 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:04.699111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:04.743267 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:04.743238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq" event={"ID":"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df","Type":"ContainerStarted","Data":"d68e18c6dfcac45debbd10bc7596dac6bff4b08fd1eb551c719d49a56a15a059"}
Jul 18 13:37:04.743392 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:04.743285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq" event={"ID":"a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df","Type":"ContainerStarted","Data":"319196cd4ca01c00e6c1665dfd3179555eca4c3ff1e98d35559667bd10c8464e"}
Jul 18 13:37:04.759420 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:04.759372    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bfvjq" podStartSLOduration=1.577328805 podStartE2EDuration="2.759357766s" totalImagesPullingTime="1.182028961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:04.758276041 +0000 UTC m=+274.414966171" watchObservedRunningTime="2026-07-18 13:37:04.759357766 +0000 UTC m=+274.416047894"
Jul 18 13:37:05.910753 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:05.910717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.163427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.163329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.176183 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.176154    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-5s54r"]
Jul 18 13:37:06.178645 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.178630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.181728 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.181706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 18 13:37:06.181979 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.181963    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 18 13:37:06.182680 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.182622    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fhldb\""
Jul 18 13:37:06.182799 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.182702    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 18 13:37:06.183021 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.182936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.183021 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.182981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccr85\" (UniqueName: \"kubernetes.io/projected/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-api-access-ccr85\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.183223 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.183078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.183223 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.183112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.183223 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.183140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.183223 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.183201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.191698 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.191678    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-5s54r"]
Jul 18 13:37:06.219723 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.219699    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-56khk"]
Jul 18 13:37:06.221652 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.221638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.223902 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.223867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 13:37:06.224016 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.223964    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 13:37:06.224016 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.224010    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 13:37:06.224166 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.224059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zg879\""
Jul 18 13:37:06.251407 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.251381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.283580 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-sys\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.283682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-textfile\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.283682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-accelerators-collector-config\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.283682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-root\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.283682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.283882 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccr85\" (UniqueName: \"kubernetes.io/projected/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-api-access-ccr85\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.284002 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.283765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6srvs\" (UniqueName: \"kubernetes.io/projected/82c3a650-589d-4ce7-82a4-c71623973731-kube-api-access-6srvs\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.284111 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.284181 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82c3a650-589d-4ce7-82a4-c71623973731-metrics-client-ca\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.284181 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.284289 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-tls\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.284289 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.284388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.284388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.284388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-wtmp\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.284542 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:37:06.284472    2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 18 13:37:06.284542 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:37:06.284520    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-tls podName:46e64f4d-3a2b-433c-82e7-1ae26cfde9dd nodeName:}" failed. No retries permitted until 2026-07-18 13:37:06.784502993 +0000 UTC m=+276.441193105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-5s54r" (UID: "46e64f4d-3a2b-433c-82e7-1ae26cfde9dd") : secret "kube-state-metrics-tls" not found
Jul 18 13:37:06.284899 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.284948 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.284893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.285216 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.285196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.286457 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.286430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.290743 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.290712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccr85\" (UniqueName: \"kubernetes.io/projected/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-api-access-ccr85\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.310969 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.310945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.384734 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-tls\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.384872 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-wtmp\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.384872 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-sys\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.384872 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-textfile\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.385019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-accelerators-collector-config\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.385019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-root\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.385019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6srvs\" (UniqueName: \"kubernetes.io/projected/82c3a650-589d-4ce7-82a4-c71623973731-kube-api-access-6srvs\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.385019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.384977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.385019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.385002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82c3a650-589d-4ce7-82a4-c71623973731-metrics-client-ca\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.385594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82c3a650-589d-4ce7-82a4-c71623973731-metrics-client-ca\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:37:06.385696    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:37:06.385746    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-tls podName:82c3a650-589d-4ce7-82a4-c71623973731 nodeName:}" failed. No retries permitted until 2026-07-18 13:37:06.885727657 +0000 UTC m=+276.542417771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-tls") pod "node-exporter-56khk" (UID: "82c3a650-589d-4ce7-82a4-c71623973731") : secret "node-exporter-tls" not found
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.386016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-wtmp\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.386090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-sys\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.386322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-textfile\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.386767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.386735    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-accelerators-collector-config\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.387196 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.386869    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/82c3a650-589d-4ce7-82a4-c71623973731-root\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.390365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.390326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.395199 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.395177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6srvs\" (UniqueName: \"kubernetes.io/projected/82c3a650-589d-4ce7-82a4-c71623973731-kube-api-access-6srvs\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.411948 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.411883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.512097 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.512014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.724320 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.724291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.789272 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.789253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.791425 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.791401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/46e64f4d-3a2b-433c-82e7-1ae26cfde9dd-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5s54r\" (UID: \"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:06.824647 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.824626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:06.890049 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.890021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-tls\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.892167 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.892146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82c3a650-589d-4ce7-82a4-c71623973731-node-exporter-tls\") pod \"node-exporter-56khk\" (UID: \"82c3a650-589d-4ce7-82a4-c71623973731\") " pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:06.924621 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:06.924597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:07.089023 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.088971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r"
Jul 18 13:37:07.129410 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.129378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:07.130174 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.130157    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-56khk"
Jul 18 13:37:07.143852 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:07.143804    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c3a650_589d_4ce7_82a4_c71623973731.slice/crio-d4045d00bc774430dc8b795b64e217e291ede556a9f44f6784b4c76af17e8243 WatchSource:0}: Error finding container d4045d00bc774430dc8b795b64e217e291ede556a9f44f6784b4c76af17e8243: Status 404 returned error can't find the container with id d4045d00bc774430dc8b795b64e217e291ede556a9f44f6784b4c76af17e8243
Jul 18 13:37:07.220446 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.220415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:07.225756 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.225736    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-5s54r"]
Jul 18 13:37:07.228256 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:07.228230    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e64f4d_3a2b_433c_82e7_1ae26cfde9dd.slice/crio-622d7938cac4419221b3da9819281823f95321023a6de60162be642a09d54fe4 WatchSource:0}: Error finding container 622d7938cac4419221b3da9819281823f95321023a6de60162be642a09d54fe4: Status 404 returned error can't find the container with id 622d7938cac4419221b3da9819281823f95321023a6de60162be642a09d54fe4
Jul 18 13:37:07.755439 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.755370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-56khk" event={"ID":"82c3a650-589d-4ce7-82a4-c71623973731","Type":"ContainerStarted","Data":"d4045d00bc774430dc8b795b64e217e291ede556a9f44f6784b4c76af17e8243"}
Jul 18 13:37:07.756674 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.756644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r" event={"ID":"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd","Type":"ContainerStarted","Data":"622d7938cac4419221b3da9819281823f95321023a6de60162be642a09d54fe4"}
Jul 18 13:37:07.777382 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:07.777347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:08.260598 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.260566    2577 patch_prober.go:28] interesting pod/image-registry-766dc69555-trrdr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 18 13:37:08.261050 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.260614    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-766dc69555-trrdr" podUID="261f05d4-c40a-4507-8664-b83ab653895f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 13:37:08.578594 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.578555    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:08.679652 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.679564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:08.763609 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.763553    2577 generic.go:356] "Generic (PLEG): container finished" podID="82c3a650-589d-4ce7-82a4-c71623973731" containerID="b7aecd5aa5527424fbeaf0040661cc5d03e53d001b2e42bb75343ad84ca888a4" exitCode=0
Jul 18 13:37:08.763758 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.763641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-56khk" event={"ID":"82c3a650-589d-4ce7-82a4-c71623973731","Type":"ContainerDied","Data":"b7aecd5aa5527424fbeaf0040661cc5d03e53d001b2e42bb75343ad84ca888a4"}
Jul 18 13:37:08.766022 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.765996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r" event={"ID":"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd","Type":"ContainerStarted","Data":"c8a1ac174695525804d309664bb9acfed8ce1be075041a8d879c0ba122694345"}
Jul 18 13:37:08.766149 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:08.766060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r" event={"ID":"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd","Type":"ContainerStarted","Data":"6fa5d89938332befa8ee2b4709bf1bdec321b9198c6fef09002425ea7695c15e"}
Jul 18 13:37:09.108135 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.107973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:09.208885 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.208852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:09.308717 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.308689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:09.645003 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.644979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:09.773771 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.773740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-56khk" event={"ID":"82c3a650-589d-4ce7-82a4-c71623973731","Type":"ContainerStarted","Data":"0034099230dcdfe0f5ba6b939750e3c842d45db6dfa24398045e62e2f4ebfdba"}
Jul 18 13:37:09.773925 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.773800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-56khk" event={"ID":"82c3a650-589d-4ce7-82a4-c71623973731","Type":"ContainerStarted","Data":"2d178fd4da8d3f7b00a083ce7ab0ff2076f9df7723f026595152e2bea95b5961"}
Jul 18 13:37:09.775743 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.775719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r" event={"ID":"46e64f4d-3a2b-433c-82e7-1ae26cfde9dd","Type":"ContainerStarted","Data":"8e882bc9f875e5338870a0062c0fba01af7dfd83df07f08214dbb09b22bbb6eb"}
Jul 18 13:37:09.792313 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.792269    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-56khk" podStartSLOduration=2.279361709 podStartE2EDuration="3.792253841s" totalImagesPullingTime="1.428254199s" totalInitContainerRuntime="84.637933ms" isStatefulPod=true podCreationTimestamp="2026-07-18 13:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:09.790528266 +0000 UTC m=+279.447218396" watchObservedRunningTime="2026-07-18 13:37:09.792253841 +0000 UTC m=+279.448943969"
Jul 18 13:37:09.808252 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:09.808209    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-5s54r" podStartSLOduration=2.462569648 podStartE2EDuration="3.808197255s" totalImagesPullingTime="1.345627607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:09.806673445 +0000 UTC m=+279.463363597" watchObservedRunningTime="2026-07-18 13:37:09.808197255 +0000 UTC m=+279.464887384"
Jul 18 13:37:10.936027 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.935995    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"]
Jul 18 13:37:10.936893 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.936853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:10.938671 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.938652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:10.940741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.940719    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rhr8f\""
Jul 18 13:37:10.940823 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.940722    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 18 13:37:10.946500 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.946209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"]
Jul 18 13:37:10.987940 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:10.987916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:11.025190 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.025166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09483280-46b3-4430-8f9c-deb20663f65d-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xvj84\" (UID: \"09483280-46b3-4430-8f9c-deb20663f65d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.025311 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.025294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfmpw\" (UniqueName: \"kubernetes.io/projected/09483280-46b3-4430-8f9c-deb20663f65d-kube-api-access-qfmpw\") pod \"monitoring-plugin-7b7b7bc8bf-xvj84\" (UID: \"09483280-46b3-4430-8f9c-deb20663f65d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.066327 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.066308    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:11.125639 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.125620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09483280-46b3-4430-8f9c-deb20663f65d-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xvj84\" (UID: \"09483280-46b3-4430-8f9c-deb20663f65d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.125729 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.125710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfmpw\" (UniqueName: \"kubernetes.io/projected/09483280-46b3-4430-8f9c-deb20663f65d-kube-api-access-qfmpw\") pod \"monitoring-plugin-7b7b7bc8bf-xvj84\" (UID: \"09483280-46b3-4430-8f9c-deb20663f65d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.128161 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.128145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09483280-46b3-4430-8f9c-deb20663f65d-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xvj84\" (UID: \"09483280-46b3-4430-8f9c-deb20663f65d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.132668 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.132646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfmpw\" (UniqueName: \"kubernetes.io/projected/09483280-46b3-4430-8f9c-deb20663f65d-kube-api-access-qfmpw\") pod \"monitoring-plugin-7b7b7bc8bf-xvj84\" (UID: \"09483280-46b3-4430-8f9c-deb20663f65d\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.216823 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.216747    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-749854b8d6-pgt2k"]
Jul 18 13:37:11.218957 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.218933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.221070 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.221032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 18 13:37:11.221168 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.221131    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 18 13:37:11.221574 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.221557    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 18 13:37:11.221704 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.221688    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jh5bg\""
Jul 18 13:37:11.221999 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.221983    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 18 13:37:11.222063 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.221989    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 18 13:37:11.222175 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.222154    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 18 13:37:11.222637 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.222619    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 18 13:37:11.225601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.225585    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 18 13:37:11.228073 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.228028    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749854b8d6-pgt2k"]
Jul 18 13:37:11.248909 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.248889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:11.326551 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-oauth-serving-cert\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.326551 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-oauth-config\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.326551 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crv8c\" (UniqueName: \"kubernetes.io/projected/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-kube-api-access-crv8c\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.326812 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-service-ca\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.326812 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-serving-cert\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.326812 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-trusted-ca-bundle\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.326812 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.326698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-config\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.381461 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.381312    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"]
Jul 18 13:37:11.383823 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:11.383795    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09483280_46b3_4430_8f9c_deb20663f65d.slice/crio-a29a3ae72f838a55fb2797f827f7bd859de7d62dd0645598588eb1c59f8e96be WatchSource:0}: Error finding container a29a3ae72f838a55fb2797f827f7bd859de7d62dd0645598588eb1c59f8e96be: Status 404 returned error can't find the container with id a29a3ae72f838a55fb2797f827f7bd859de7d62dd0645598588eb1c59f8e96be
Jul 18 13:37:11.427286 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-trusted-ca-bundle\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.427393 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-config\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.427393 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-oauth-serving-cert\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.427393 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-oauth-config\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.427550 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crv8c\" (UniqueName: \"kubernetes.io/projected/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-kube-api-access-crv8c\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.427602 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-service-ca\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.427650 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.427631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-serving-cert\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.428135 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.428108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-config\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.428251 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.428108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-oauth-serving-cert\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.428251 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.428238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-trusted-ca-bundle\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.428336 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.428280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-service-ca\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.429645 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.429629    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-oauth-config\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.429864 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.429848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-serving-cert\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.434420 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.434397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crv8c\" (UniqueName: \"kubernetes.io/projected/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-kube-api-access-crv8c\") pod \"console-749854b8d6-pgt2k\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") " pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.529772 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.529747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:11.650323 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.650296    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749854b8d6-pgt2k"]
Jul 18 13:37:11.652020 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:11.651987    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a950837_ed47_41b7_bfe2_abe4fcb2ce44.slice/crio-ae3490d0324512e577ab61c539ddb96d99cb745f8b78904149f30d220fca48f4 WatchSource:0}: Error finding container ae3490d0324512e577ab61c539ddb96d99cb745f8b78904149f30d220fca48f4: Status 404 returned error can't find the container with id ae3490d0324512e577ab61c539ddb96d99cb745f8b78904149f30d220fca48f4
Jul 18 13:37:11.784102 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.783996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84" event={"ID":"09483280-46b3-4430-8f9c-deb20663f65d","Type":"ContainerStarted","Data":"a29a3ae72f838a55fb2797f827f7bd859de7d62dd0645598588eb1c59f8e96be"}
Jul 18 13:37:11.785085 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:11.785062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749854b8d6-pgt2k" event={"ID":"7a950837-ed47-41b7-bfe2-abe4fcb2ce44","Type":"ContainerStarted","Data":"ae3490d0324512e577ab61c539ddb96d99cb745f8b78904149f30d220fca48f4"}
Jul 18 13:37:12.353343 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:37:12.353300    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9j6vs" podUID="894a67ce-7ec2-4ffb-b31a-3b62e845a2f4"
Jul 18 13:37:12.387275 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:12.387240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:12.790425 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:12.790289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:37:13.090826 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.090736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:13.191520 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.191488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:13.283457 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.283421    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-766dc69555-trrdr" podUID="261f05d4-c40a-4507-8664-b83ab653895f" containerName="registry" containerID="cri-o://5c553339584dd6c76e64cf1cd18e5f9f361f4d4c0d7ff61b108f597b6e4ced0e" gracePeriod=30
Jul 18 13:37:13.796982 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.796945    2577 generic.go:356] "Generic (PLEG): container finished" podID="261f05d4-c40a-4507-8664-b83ab653895f" containerID="5c553339584dd6c76e64cf1cd18e5f9f361f4d4c0d7ff61b108f597b6e4ced0e" exitCode=0
Jul 18 13:37:13.797461 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.797025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766dc69555-trrdr" event={"ID":"261f05d4-c40a-4507-8664-b83ab653895f","Type":"ContainerDied","Data":"5c553339584dd6c76e64cf1cd18e5f9f361f4d4c0d7ff61b108f597b6e4ced0e"}
Jul 18 13:37:13.798702 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.798675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84" event={"ID":"09483280-46b3-4430-8f9c-deb20663f65d","Type":"ContainerStarted","Data":"e643e785dbf5351213267ba2523ef8bcd77f52cdef8baa0e208b0d10afad9526"}
Jul 18 13:37:13.799009 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.798964    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:13.804869 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.804848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84"
Jul 18 13:37:13.813840 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:13.813802    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xvj84" podStartSLOduration=2.294204609 podStartE2EDuration="3.813788914s" totalImagesPullingTime="1.519584305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:13.813182299 +0000 UTC m=+283.469872428" watchObservedRunningTime="2026-07-18 13:37:13.813788914 +0000 UTC m=+283.470479042"
Jul 18 13:37:14.503859 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.503837    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:37:14.554068 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554031    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-bound-sa-token\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554102    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-registry-certificates\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554161 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554133    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnxcj\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-kube-api-access-jnxcj\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554282 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554263    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554345 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-installation-pull-secrets\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554394 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-trusted-ca\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554394 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554382    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/261f05d4-c40a-4507-8664-b83ab653895f-ca-trust-extracted\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554484 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554435    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-image-registry-private-configuration\") pod \"261f05d4-c40a-4507-8664-b83ab653895f\" (UID: \"261f05d4-c40a-4507-8664-b83ab653895f\") "
Jul 18 13:37:14.554484 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554466    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:14.554716 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554682    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-registry-certificates\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.554978 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.554908    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:14.556777 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.556746    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:14.556941 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.556919    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-kube-api-access-jnxcj" (OuterVolumeSpecName: "kube-api-access-jnxcj") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "kube-api-access-jnxcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:14.557274 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.557250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:14.557350 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.557270    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:14.557350 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.557324    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:14.563702 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.563673    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261f05d4-c40a-4507-8664-b83ab653895f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "261f05d4-c40a-4507-8664-b83ab653895f" (UID: "261f05d4-c40a-4507-8664-b83ab653895f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:37:14.655388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655346    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-registry-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.655388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655371    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-installation-pull-secrets\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.655388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655385    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/261f05d4-c40a-4507-8664-b83ab653895f-trusted-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.655388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655396    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/261f05d4-c40a-4507-8664-b83ab653895f-ca-trust-extracted\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.655701 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655406    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/261f05d4-c40a-4507-8664-b83ab653895f-image-registry-private-configuration\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.655701 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655415    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-bound-sa-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.655701 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.655423    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnxcj\" (UniqueName: \"kubernetes.io/projected/261f05d4-c40a-4507-8664-b83ab653895f-kube-api-access-jnxcj\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:14.803725 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.803692    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766dc69555-trrdr"
Jul 18 13:37:14.804185 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.803724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766dc69555-trrdr" event={"ID":"261f05d4-c40a-4507-8664-b83ab653895f","Type":"ContainerDied","Data":"20eeb5a339faeda47483e80ebb547f8627e391d8978c4a70adda211ed43c4e38"}
Jul 18 13:37:14.804185 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.803777    2577 scope.go:117] "RemoveContainer" containerID="5c553339584dd6c76e64cf1cd18e5f9f361f4d4c0d7ff61b108f597b6e4ced0e"
Jul 18 13:37:14.805489 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.805453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749854b8d6-pgt2k" event={"ID":"7a950837-ed47-41b7-bfe2-abe4fcb2ce44","Type":"ContainerStarted","Data":"dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1"}
Jul 18 13:37:14.823110 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.823067    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-749854b8d6-pgt2k" podStartSLOduration=0.939153067 podStartE2EDuration="3.823030332s" totalImagesPullingTime="2.883877265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:14.822274202 +0000 UTC m=+284.478964332" watchObservedRunningTime="2026-07-18 13:37:14.823030332 +0000 UTC m=+284.479720452"
Jul 18 13:37:14.834543 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.834518    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-766dc69555-trrdr"]
Jul 18 13:37:14.838048 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:14.838010    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-766dc69555-trrdr"]
Jul 18 13:37:16.169026 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.168990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:37:16.171355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.171333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/894a67ce-7ec2-4ffb-b31a-3b62e845a2f4-metrics-tls\") pod \"dns-default-9j6vs\" (UID: \"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4\") " pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:37:16.393441 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.393406    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t94bf\""
Jul 18 13:37:16.401434 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.401404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:37:16.548222 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.548127    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9j6vs"]
Jul 18 13:37:16.550431 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:16.550406    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894a67ce_7ec2_4ffb_b31a_3b62e845a2f4.slice/crio-80d5b1b761cab86e3e9ae4d7fa2a778072f0a2990360cb0011eaa59db0cd369f WatchSource:0}: Error finding container 80d5b1b761cab86e3e9ae4d7fa2a778072f0a2990360cb0011eaa59db0cd369f: Status 404 returned error can't find the container with id 80d5b1b761cab86e3e9ae4d7fa2a778072f0a2990360cb0011eaa59db0cd369f
Jul 18 13:37:16.702606 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.702576    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f55595c9-ftfhd"]
Jul 18 13:37:16.702926 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.702911    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261f05d4-c40a-4507-8664-b83ab653895f" containerName="registry"
Jul 18 13:37:16.702996 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.702930    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="261f05d4-c40a-4507-8664-b83ab653895f" containerName="registry"
Jul 18 13:37:16.703103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.703006    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="261f05d4-c40a-4507-8664-b83ab653895f" containerName="registry"
Jul 18 13:37:16.707263 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.707242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.716017 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.715992    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f55595c9-ftfhd"]
Jul 18 13:37:16.774767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-serving-cert\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.774767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-config\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.775015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-trusted-ca-bundle\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.775015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-service-ca\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.775015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcbt7\" (UniqueName: \"kubernetes.io/projected/ee34af3b-6d54-424f-bd61-a1d9732d8554-kube-api-access-bcbt7\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.775015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-oauth-config\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.775015 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.774955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-oauth-serving-cert\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.801351 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.801316    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261f05d4-c40a-4507-8664-b83ab653895f" path="/var/lib/kubelet/pods/261f05d4-c40a-4507-8664-b83ab653895f/volumes"
Jul 18 13:37:16.814637 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.814595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9j6vs" event={"ID":"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4","Type":"ContainerStarted","Data":"80d5b1b761cab86e3e9ae4d7fa2a778072f0a2990360cb0011eaa59db0cd369f"}
Jul 18 13:37:16.876192 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876155    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-serving-cert\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.876380 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-config\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.876380 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-trusted-ca-bundle\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.876380 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-service-ca\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.876380 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcbt7\" (UniqueName: \"kubernetes.io/projected/ee34af3b-6d54-424f-bd61-a1d9732d8554-kube-api-access-bcbt7\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.876585 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-oauth-config\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.876585 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.876408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-oauth-serving-cert\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.877102 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.877078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-config\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.877213 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.877166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-service-ca\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.877288 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.877266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-trusted-ca-bundle\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.877332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.877293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-oauth-serving-cert\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.878817 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.878801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-serving-cert\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.878986 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.878964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-oauth-config\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:16.883172 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:16.883153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcbt7\" (UniqueName: \"kubernetes.io/projected/ee34af3b-6d54-424f-bd61-a1d9732d8554-kube-api-access-bcbt7\") pod \"console-56f55595c9-ftfhd\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") " pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:17.017079 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:17.016976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:17.189163 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:17.189070    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f55595c9-ftfhd"]
Jul 18 13:37:17.192431 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:37:17.192401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee34af3b_6d54_424f_bd61_a1d9732d8554.slice/crio-34a4e008a294373ce2f84eab4acc0379c0f0ac747d2c2d2dc50a5234db0cef71 WatchSource:0}: Error finding container 34a4e008a294373ce2f84eab4acc0379c0f0ac747d2c2d2dc50a5234db0cef71: Status 404 returned error can't find the container with id 34a4e008a294373ce2f84eab4acc0379c0f0ac747d2c2d2dc50a5234db0cef71
Jul 18 13:37:17.824668 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:17.824632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f55595c9-ftfhd" event={"ID":"ee34af3b-6d54-424f-bd61-a1d9732d8554","Type":"ContainerStarted","Data":"bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69"}
Jul 18 13:37:17.824919 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:17.824717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f55595c9-ftfhd" event={"ID":"ee34af3b-6d54-424f-bd61-a1d9732d8554","Type":"ContainerStarted","Data":"34a4e008a294373ce2f84eab4acc0379c0f0ac747d2c2d2dc50a5234db0cef71"}
Jul 18 13:37:17.841989 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:17.841937    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f55595c9-ftfhd" podStartSLOduration=1.841917417 podStartE2EDuration="1.841917417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:17.839464478 +0000 UTC m=+287.496154599" watchObservedRunningTime="2026-07-18 13:37:17.841917417 +0000 UTC m=+287.498607548"
Jul 18 13:37:18.830716 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:18.830681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9j6vs" event={"ID":"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4","Type":"ContainerStarted","Data":"308dc4b5e45e37142fa5c4429506c52b40b831d7a75263254bc7982827f9eadf"}
Jul 18 13:37:18.831146 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:18.830743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9j6vs" event={"ID":"894a67ce-7ec2-4ffb-b31a-3b62e845a2f4","Type":"ContainerStarted","Data":"b4cdd12631c8011dfb400208d848e63a36a01f10ffd0e2608ab01866896b67b9"}
Jul 18 13:37:18.831146 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:18.831011    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:37:18.847476 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:18.847435    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9j6vs" podStartSLOduration=251.358016422 podStartE2EDuration="4m12.847421859s" totalImagesPullingTime="1.489405437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:37:18.845652433 +0000 UTC m=+288.502342564" watchObservedRunningTime="2026-07-18 13:37:18.847421859 +0000 UTC m=+288.504111987"
Jul 18 13:37:19.961262 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:19.961234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:20.011246 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:20.011216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:20.088820 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:20.088786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:20.187721 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:20.187687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:20.890133 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:20.890094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:20.992485 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:20.992450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:21.286941 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.286911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:21.344197 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.344163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:21.530447 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.530408    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:21.530696 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.530681    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:21.533975 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.533936    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:21.536104 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.536083    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:21.590988 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.590897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:21.847260 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:21.847183    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:22.810529 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:22.810498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:23.599899 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:23.599860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:23.651329 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:23.651278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:26.896016 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:26.895980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:26.994195 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:26.994162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:27.017925 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:27.017897    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:27.018082 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:27.017966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:27.022361 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:27.022336    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:27.869497 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:27.869467    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:37:27.914005 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:27.913972    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749854b8d6-pgt2k"]
Jul 18 13:37:28.009422 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:28.009394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:28.110637 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:28.110589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:28.837029 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:28.837002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9j6vs"
Jul 18 13:37:30.697197 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:30.697175    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:37:30.697541 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:30.697202    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:37:30.704179 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:30.704159    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:37:30.704682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:30.704656    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:37:30.708171 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:30.708154    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 13:37:32.727704 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:32.727623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:32.820144 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:32.820111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:35.882467 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:35.882425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:35.982130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:35.982089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:37.966441 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:37.966409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:38.021402 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:38.021370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:42.926022 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:42.925986    2577 generic.go:356] "Generic (PLEG): container finished" podID="9e57eaa4-6de2-498b-830c-690e2fdce624" containerID="e088acf6dcfb78ef23984d1db2f3159b2c4a07a4fa8920000f4359f76dab57d9" exitCode=0
Jul 18 13:37:42.926441 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:42.926061    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-5nkpc" event={"ID":"9e57eaa4-6de2-498b-830c-690e2fdce624","Type":"ContainerDied","Data":"e088acf6dcfb78ef23984d1db2f3159b2c4a07a4fa8920000f4359f76dab57d9"}
Jul 18 13:37:42.926482 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:42.926455    2577 scope.go:117] "RemoveContainer" containerID="e088acf6dcfb78ef23984d1db2f3159b2c4a07a4fa8920000f4359f76dab57d9"
Jul 18 13:37:43.931300 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:43.931264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-5nkpc" event={"ID":"9e57eaa4-6de2-498b-830c-690e2fdce624","Type":"ContainerStarted","Data":"a2657a6c82f9d143820e20411f1e13bb37ec19644d5dabce9ced30c6884c6585"}
Jul 18 13:37:46.944383 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:46.944348    2577 generic.go:356] "Generic (PLEG): container finished" podID="b59163b6-7d4a-4c22-ae0c-98ffaefc7f35" containerID="2f7fa29030c655a2f08478901a2d10642ff9a59dd7d076652146469b9225e4e9" exitCode=0
Jul 18 13:37:46.944769 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:46.944425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r" event={"ID":"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35","Type":"ContainerDied","Data":"2f7fa29030c655a2f08478901a2d10642ff9a59dd7d076652146469b9225e4e9"}
Jul 18 13:37:46.944830 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:46.944815    2577 scope.go:117] "RemoveContainer" containerID="2f7fa29030c655a2f08478901a2d10642ff9a59dd7d076652146469b9225e4e9"
Jul 18 13:37:47.950098 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:47.950066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jmc6r" event={"ID":"b59163b6-7d4a-4c22-ae0c-98ffaefc7f35","Type":"ContainerStarted","Data":"709e1668817bd3ef0f3f3f85af2b9a8f460f427de22aeed97b9b7586892fed24"}
Jul 18 13:37:50.884621 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:50.884560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:50.934191 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:50.934153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:37:52.935856 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:52.935789    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-749854b8d6-pgt2k" podUID="7a950837-ed47-41b7-bfe2-abe4fcb2ce44" containerName="console" containerID="cri-o://dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1" gracePeriod=15
Jul 18 13:37:53.185098 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.185074    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:53.290459 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290423    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-oauth-config\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290653 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-service-ca\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290653 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290497    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-config\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290653 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-trusted-ca-bundle\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290653 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290554    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-serving-cert\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290653 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290593    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crv8c\" (UniqueName: \"kubernetes.io/projected/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-kube-api-access-crv8c\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290653 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-oauth-serving-cert\") pod \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\" (UID: \"7a950837-ed47-41b7-bfe2-abe4fcb2ce44\") "
Jul 18 13:37:53.290966 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290851    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-config" (OuterVolumeSpecName: "console-config") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:53.291026 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.290960    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-service-ca" (OuterVolumeSpecName: "service-ca") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:53.291193 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.291167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:53.291249 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.291208    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:37:53.292847 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.292822    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:53.292910 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.292844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:37:53.292910 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.292880    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-kube-api-access-crv8c" (OuterVolumeSpecName: "kube-api-access-crv8c") pod "7a950837-ed47-41b7-bfe2-abe4fcb2ce44" (UID: "7a950837-ed47-41b7-bfe2-abe4fcb2ce44"). InnerVolumeSpecName "kube-api-access-crv8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:37:53.391481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391437    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-oauth-config\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.391481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391473    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.391481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391486    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-config\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.391711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391497    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-trusted-ca-bundle\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.391711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391509    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-console-serving-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.391711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391520    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crv8c\" (UniqueName: \"kubernetes.io/projected/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-kube-api-access-crv8c\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.391711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.391531    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a950837-ed47-41b7-bfe2-abe4fcb2ce44-oauth-serving-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:37:53.975906 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.975874    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a950837-ed47-41b7-bfe2-abe4fcb2ce44" containerID="dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1" exitCode=0
Jul 18 13:37:53.976387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.975944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749854b8d6-pgt2k" event={"ID":"7a950837-ed47-41b7-bfe2-abe4fcb2ce44","Type":"ContainerDied","Data":"dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1"}
Jul 18 13:37:53.976387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.976011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749854b8d6-pgt2k" event={"ID":"7a950837-ed47-41b7-bfe2-abe4fcb2ce44","Type":"ContainerDied","Data":"ae3490d0324512e577ab61c539ddb96d99cb745f8b78904149f30d220fca48f4"}
Jul 18 13:37:53.976387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.976062    2577 scope.go:117] "RemoveContainer" containerID="dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1"
Jul 18 13:37:53.976387 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.975925    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749854b8d6-pgt2k"
Jul 18 13:37:53.987118 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.987097    2577 scope.go:117] "RemoveContainer" containerID="dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1"
Jul 18 13:37:53.987405 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:37:53.987383    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1\": container with ID starting with dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1 not found: ID does not exist" containerID="dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1"
Jul 18 13:37:53.987478 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.987413    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1"} err="failed to get container status \"dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1\": rpc error: code = NotFound desc = could not find container \"dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1\": container with ID starting with dce91280b771a4c488c677437e608b669baee6d04fbf05c78207f8084ad9a8c1 not found: ID does not exist"
Jul 18 13:37:53.998950 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:53.998919    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749854b8d6-pgt2k"]
Jul 18 13:37:54.003494 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:54.003460    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-749854b8d6-pgt2k"]
Jul 18 13:37:54.801220 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:37:54.801186    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a950837-ed47-41b7-bfe2-abe4fcb2ce44" path="/var/lib/kubelet/pods/7a950837-ed47-41b7-bfe2-abe4fcb2ce44/volumes"
Jul 18 13:38:22.246060 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:22.245998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:22.295345 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:22.295315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:28.345428 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:28.345385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:28.448767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:28.448731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:34.090194 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:34.090158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:34.144805 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:34.144767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:49.569187 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:49.569144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:49.622326 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:49.622293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:49.874572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:49.874470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:49.925973 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:49.925937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:50.480863 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:50.480827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:50.529821 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:50.528270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:50.885358 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:50.885309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:38:50.987097 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:38:50.987057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:00.337481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:00.337442    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f55595c9-ftfhd"]
Jul 18 13:39:00.442915 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:00.442883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:00.492705 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:00.492671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:00.547018 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:00.546983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:13.903517 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:13.903481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:14.004945 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:14.004910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:25.361058 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.360983    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56f55595c9-ftfhd" podUID="ee34af3b-6d54-424f-bd61-a1d9732d8554" containerName="console" containerID="cri-o://bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69" gracePeriod=15
Jul 18 13:39:25.608394 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.608370    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:39:25.716859 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.716766    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-config\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.716859 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.716823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-oauth-config\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.716859 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.716848    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-oauth-serving-cert\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.717162 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.716980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcbt7\" (UniqueName: \"kubernetes.io/projected/ee34af3b-6d54-424f-bd61-a1d9732d8554-kube-api-access-bcbt7\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.717162 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717018    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-service-ca\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.717162 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717069    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-trusted-ca-bundle\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.717162 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-serving-cert\") pod \"ee34af3b-6d54-424f-bd61-a1d9732d8554\" (UID: \"ee34af3b-6d54-424f-bd61-a1d9732d8554\") "
Jul 18 13:39:25.717364 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:39:25.717364 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717206    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-config" (OuterVolumeSpecName: "console-config") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:39:25.717455 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717367    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-oauth-serving-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:25.717455 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717411    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-service-ca" (OuterVolumeSpecName: "service-ca") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:39:25.717529 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.717418    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 13:39:25.719243 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.719217    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:39:25.719243 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.719233    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:39:25.719376 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.719246    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee34af3b-6d54-424f-bd61-a1d9732d8554-kube-api-access-bcbt7" (OuterVolumeSpecName: "kube-api-access-bcbt7") pod "ee34af3b-6d54-424f-bd61-a1d9732d8554" (UID: "ee34af3b-6d54-424f-bd61-a1d9732d8554"). InnerVolumeSpecName "kube-api-access-bcbt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:39:25.818540 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.818506    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-oauth-config\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:25.818540 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.818535    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcbt7\" (UniqueName: \"kubernetes.io/projected/ee34af3b-6d54-424f-bd61-a1d9732d8554-kube-api-access-bcbt7\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:25.818540 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.818544    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:25.818761 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.818553    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-trusted-ca-bundle\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:25.818761 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.818563    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-serving-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:25.818761 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:25.818571    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee34af3b-6d54-424f-bd61-a1d9732d8554-console-config\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:39:26.326356 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.326323    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee34af3b-6d54-424f-bd61-a1d9732d8554" containerID="bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69" exitCode=0
Jul 18 13:39:26.326526 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.326376    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f55595c9-ftfhd"
Jul 18 13:39:26.326526 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.326409    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f55595c9-ftfhd" event={"ID":"ee34af3b-6d54-424f-bd61-a1d9732d8554","Type":"ContainerDied","Data":"bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69"}
Jul 18 13:39:26.326526 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.326462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f55595c9-ftfhd" event={"ID":"ee34af3b-6d54-424f-bd61-a1d9732d8554","Type":"ContainerDied","Data":"34a4e008a294373ce2f84eab4acc0379c0f0ac747d2c2d2dc50a5234db0cef71"}
Jul 18 13:39:26.326526 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.326490    2577 scope.go:117] "RemoveContainer" containerID="bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69"
Jul 18 13:39:26.335025 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.335008    2577 scope.go:117] "RemoveContainer" containerID="bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69"
Jul 18 13:39:26.335270 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:39:26.335252    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69\": container with ID starting with bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69 not found: ID does not exist" containerID="bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69"
Jul 18 13:39:26.335321 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.335278    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69"} err="failed to get container status \"bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69\": rpc error: code = NotFound desc = could not find container \"bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69\": container with ID starting with bcf3568ec462500df039ac95c13e7875e4204d848f4ae5a0880bff0cbdecee69 not found: ID does not exist"
Jul 18 13:39:26.346646 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.346620    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f55595c9-ftfhd"]
Jul 18 13:39:26.349834 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.349811    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f55595c9-ftfhd"]
Jul 18 13:39:26.804690 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:26.804651    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee34af3b-6d54-424f-bd61-a1d9732d8554" path="/var/lib/kubelet/pods/ee34af3b-6d54-424f-bd61-a1d9732d8554/volumes"
Jul 18 13:39:28.644862 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:28.644792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:28.747007 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:28.746972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:28.875774 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:28.875734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:28.976663 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:28.976553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:50.475116 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:50.475071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:51.474412 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:51.474371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:54.089118 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:54.089014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:54.143792 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:54.143763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:55.639883 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.639850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:55.736969 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.736941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:55.743500 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743472    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9xnk7"]
Jul 18 13:39:55.743765 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743754    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee34af3b-6d54-424f-bd61-a1d9732d8554" containerName="console"
Jul 18 13:39:55.743813 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743767    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee34af3b-6d54-424f-bd61-a1d9732d8554" containerName="console"
Jul 18 13:39:55.743813 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743783    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a950837-ed47-41b7-bfe2-abe4fcb2ce44" containerName="console"
Jul 18 13:39:55.743813 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743790    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a950837-ed47-41b7-bfe2-abe4fcb2ce44" containerName="console"
Jul 18 13:39:55.743967 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743833    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a950837-ed47-41b7-bfe2-abe4fcb2ce44" containerName="console"
Jul 18 13:39:55.743967 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.743841    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee34af3b-6d54-424f-bd61-a1d9732d8554" containerName="console"
Jul 18 13:39:55.746617 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.746602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:55.749234 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.749217    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-btqcf\""
Jul 18 13:39:55.750034 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.750021    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 18 13:39:55.751382 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.751368    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 18 13:39:55.771360 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.771337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9xnk7"]
Jul 18 13:39:55.850292 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.850261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td7pf\" (UniqueName: \"kubernetes.io/projected/45a11ae4-b1d3-40a5-9622-ed95dc898069-kube-api-access-td7pf\") pod \"cert-manager-webhook-6486ffd997-9xnk7\" (UID: \"45a11ae4-b1d3-40a5-9622-ed95dc898069\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:55.850442 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.850299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45a11ae4-b1d3-40a5-9622-ed95dc898069-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9xnk7\" (UID: \"45a11ae4-b1d3-40a5-9622-ed95dc898069\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:55.912707 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.912620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:55.950668 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.950640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td7pf\" (UniqueName: \"kubernetes.io/projected/45a11ae4-b1d3-40a5-9622-ed95dc898069-kube-api-access-td7pf\") pod \"cert-manager-webhook-6486ffd997-9xnk7\" (UID: \"45a11ae4-b1d3-40a5-9622-ed95dc898069\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:55.950793 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.950676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45a11ae4-b1d3-40a5-9622-ed95dc898069-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9xnk7\" (UID: \"45a11ae4-b1d3-40a5-9622-ed95dc898069\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:55.965361 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.965331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45a11ae4-b1d3-40a5-9622-ed95dc898069-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9xnk7\" (UID: \"45a11ae4-b1d3-40a5-9622-ed95dc898069\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:55.965524 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:55.965502    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td7pf\" (UniqueName: \"kubernetes.io/projected/45a11ae4-b1d3-40a5-9622-ed95dc898069-kube-api-access-td7pf\") pod \"cert-manager-webhook-6486ffd997-9xnk7\" (UID: \"45a11ae4-b1d3-40a5-9622-ed95dc898069\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:56.017531 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.017496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:56.055529 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.055488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:56.185313 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.185156    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9xnk7"]
Jul 18 13:39:56.187792 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:39:56.187765    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45a11ae4_b1d3_40a5_9622_ed95dc898069.slice/crio-36a78f1df61db2d89369b1cbd26743288fbedae0f4f60d7ef519be10bd56190e WatchSource:0}: Error finding container 36a78f1df61db2d89369b1cbd26743288fbedae0f4f60d7ef519be10bd56190e: Status 404 returned error can't find the container with id 36a78f1df61db2d89369b1cbd26743288fbedae0f4f60d7ef519be10bd56190e
Jul 18 13:39:56.189692 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.189672    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 13:39:56.434574 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.434539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7" event={"ID":"45a11ae4-b1d3-40a5-9622-ed95dc898069","Type":"ContainerStarted","Data":"36a78f1df61db2d89369b1cbd26743288fbedae0f4f60d7ef519be10bd56190e"}
Jul 18 13:39:56.723331 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.723294    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"]
Jul 18 13:39:56.727103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.727069    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.729411 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.729385    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 18 13:39:56.729411 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.729400    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kfclk\""
Jul 18 13:39:56.729679 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.729662    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 18 13:39:56.735472 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.735444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"]
Jul 18 13:39:56.858876 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.858835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4jrg\" (UniqueName: \"kubernetes.io/projected/aee1162a-ea93-4c08-959c-227fabca6e72-kube-api-access-j4jrg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.859155 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.858893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.859155 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.858985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.960253 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.960157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.960253 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.960266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4jrg\" (UniqueName: \"kubernetes.io/projected/aee1162a-ea93-4c08-959c-227fabca6e72-kube-api-access-j4jrg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.960253 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.960301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.960674 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.960617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.960796 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.960674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:56.968809 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:56.968779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4jrg\" (UniqueName: \"kubernetes.io/projected/aee1162a-ea93-4c08-959c-227fabca6e72-kube-api-access-j4jrg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:57.040617 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:57.040570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:39:57.200081 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:57.199999    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"]
Jul 18 13:39:57.205161 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:39:57.205109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee1162a_ea93_4c08_959c_227fabca6e72.slice/crio-ca8954461143c55134cbfbd57ec217f544a3499351a65afaab9f0980ebac1000 WatchSource:0}: Error finding container ca8954461143c55134cbfbd57ec217f544a3499351a65afaab9f0980ebac1000: Status 404 returned error can't find the container with id ca8954461143c55134cbfbd57ec217f544a3499351a65afaab9f0980ebac1000
Jul 18 13:39:57.441775 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:57.441667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t" event={"ID":"aee1162a-ea93-4c08-959c-227fabca6e72","Type":"ContainerStarted","Data":"ca8954461143c55134cbfbd57ec217f544a3499351a65afaab9f0980ebac1000"}
Jul 18 13:39:58.035964 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:58.035410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:58.288192 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:58.288112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:39:59.451676 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:59.451595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7" event={"ID":"45a11ae4-b1d3-40a5-9622-ed95dc898069","Type":"ContainerStarted","Data":"cfeaf30fc1711b7df7fc2b029bc458edb1ee8dac55c369544c7d05c987646384"}
Jul 18 13:39:59.452107 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:59.451729    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:39:59.467802 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:39:59.467742    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7" podStartSLOduration=1.583219836 podStartE2EDuration="4.467727528s" totalImagesPullingTime="2.884507692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:39:59.466608025 +0000 UTC m=+449.123298155" watchObservedRunningTime="2026-07-18 13:39:59.467727528 +0000 UTC m=+449.124417657"
Jul 18 13:40:00.879225 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:00.879191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:00.979880 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:00.979839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:04.474865 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:04.474778    2577 generic.go:356] "Generic (PLEG): container finished" podID="aee1162a-ea93-4c08-959c-227fabca6e72" containerID="2b1de59115976542efd9eeacfd42a78077ecad18bba8d6b187a5bee3ba321b39" exitCode=0
Jul 18 13:40:04.475292 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:04.474862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t" event={"ID":"aee1162a-ea93-4c08-959c-227fabca6e72","Type":"ContainerDied","Data":"2b1de59115976542efd9eeacfd42a78077ecad18bba8d6b187a5bee3ba321b39"}
Jul 18 13:40:05.385421 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:05.385378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:05.459437 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:05.459405    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9xnk7"
Jul 18 13:40:05.997875 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:05.997839    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8w9hg"]
Jul 18 13:40:06.008172 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.008138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.009728 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.009704    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8w9hg"]
Jul 18 13:40:06.010233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.010208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4nh87\""
Jul 18 13:40:06.143350 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.143316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05f21bbd-e4a7-4c39-b728-afe9319432b9-bound-sa-token\") pod \"cert-manager-55b4d954d5-8w9hg\" (UID: \"05f21bbd-e4a7-4c39-b728-afe9319432b9\") " pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.143538 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.143458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvdmd\" (UniqueName: \"kubernetes.io/projected/05f21bbd-e4a7-4c39-b728-afe9319432b9-kube-api-access-nvdmd\") pod \"cert-manager-55b4d954d5-8w9hg\" (UID: \"05f21bbd-e4a7-4c39-b728-afe9319432b9\") " pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.245611 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.245578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvdmd\" (UniqueName: \"kubernetes.io/projected/05f21bbd-e4a7-4c39-b728-afe9319432b9-kube-api-access-nvdmd\") pod \"cert-manager-55b4d954d5-8w9hg\" (UID: \"05f21bbd-e4a7-4c39-b728-afe9319432b9\") " pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.245788 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.245630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05f21bbd-e4a7-4c39-b728-afe9319432b9-bound-sa-token\") pod \"cert-manager-55b4d954d5-8w9hg\" (UID: \"05f21bbd-e4a7-4c39-b728-afe9319432b9\") " pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.254575 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.254507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05f21bbd-e4a7-4c39-b728-afe9319432b9-bound-sa-token\") pod \"cert-manager-55b4d954d5-8w9hg\" (UID: \"05f21bbd-e4a7-4c39-b728-afe9319432b9\") " pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.254703 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.254600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvdmd\" (UniqueName: \"kubernetes.io/projected/05f21bbd-e4a7-4c39-b728-afe9319432b9-kube-api-access-nvdmd\") pod \"cert-manager-55b4d954d5-8w9hg\" (UID: \"05f21bbd-e4a7-4c39-b728-afe9319432b9\") " pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.319609 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.319581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8w9hg"
Jul 18 13:40:06.453247 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.453219    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8w9hg"]
Jul 18 13:40:06.485695 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.485665    2577 generic.go:356] "Generic (PLEG): container finished" podID="aee1162a-ea93-4c08-959c-227fabca6e72" containerID="7e89a7714b4c177f25e74cce791b643b4851233113f706deec18d4ede3b6ee3c" exitCode=0
Jul 18 13:40:06.485812 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:06.485748    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t" event={"ID":"aee1162a-ea93-4c08-959c-227fabca6e72","Type":"ContainerDied","Data":"7e89a7714b4c177f25e74cce791b643b4851233113f706deec18d4ede3b6ee3c"}
Jul 18 13:40:06.503415 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:40:06.503384    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05f21bbd_e4a7_4c39_b728_afe9319432b9.slice/crio-95a88c030e9b20f4236f99f4bb70a3a8720ec43b14ad74a92549d1efc4a618a6 WatchSource:0}: Error finding container 95a88c030e9b20f4236f99f4bb70a3a8720ec43b14ad74a92549d1efc4a618a6: Status 404 returned error can't find the container with id 95a88c030e9b20f4236f99f4bb70a3a8720ec43b14ad74a92549d1efc4a618a6
Jul 18 13:40:07.491568 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:07.491533    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8w9hg" event={"ID":"05f21bbd-e4a7-4c39-b728-afe9319432b9","Type":"ContainerStarted","Data":"7df26f1e02030e5ff9f5e8a7a6f0782f351cef331d3fb7a981d1900ea6cc8f72"}
Jul 18 13:40:07.491933 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:07.491576    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8w9hg" event={"ID":"05f21bbd-e4a7-4c39-b728-afe9319432b9","Type":"ContainerStarted","Data":"95a88c030e9b20f4236f99f4bb70a3a8720ec43b14ad74a92549d1efc4a618a6"}
Jul 18 13:40:07.507018 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:07.506974    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8w9hg" podStartSLOduration=2.5069581640000003 podStartE2EDuration="2.506958164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:40:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:40:07.505581409 +0000 UTC m=+457.162271538" watchObservedRunningTime="2026-07-18 13:40:07.506958164 +0000 UTC m=+457.163648293"
Jul 18 13:40:13.517960 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:13.517931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t" event={"ID":"aee1162a-ea93-4c08-959c-227fabca6e72","Type":"ContainerStarted","Data":"bd729b0360d40c8fb959bca4eae33321d79875c71016307dc7a894a84590f81f"}
Jul 18 13:40:14.523679 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.523647    2577 generic.go:356] "Generic (PLEG): container finished" podID="aee1162a-ea93-4c08-959c-227fabca6e72" containerID="bd729b0360d40c8fb959bca4eae33321d79875c71016307dc7a894a84590f81f" exitCode=0
Jul 18 13:40:14.524103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.523719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t" event={"ID":"aee1162a-ea93-4c08-959c-227fabca6e72","Type":"ContainerDied","Data":"bd729b0360d40c8fb959bca4eae33321d79875c71016307dc7a894a84590f81f"}
Jul 18 13:40:14.659959 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.659935    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:40:14.715026 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.714988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-util\") pod \"aee1162a-ea93-4c08-959c-227fabca6e72\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") "
Jul 18 13:40:14.715236 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.715109    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4jrg\" (UniqueName: \"kubernetes.io/projected/aee1162a-ea93-4c08-959c-227fabca6e72-kube-api-access-j4jrg\") pod \"aee1162a-ea93-4c08-959c-227fabca6e72\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") "
Jul 18 13:40:14.715236 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.715180    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-bundle\") pod \"aee1162a-ea93-4c08-959c-227fabca6e72\" (UID: \"aee1162a-ea93-4c08-959c-227fabca6e72\") "
Jul 18 13:40:14.715582 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.715552    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-bundle" (OuterVolumeSpecName: "bundle") pod "aee1162a-ea93-4c08-959c-227fabca6e72" (UID: "aee1162a-ea93-4c08-959c-227fabca6e72"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:40:14.717234 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.717206    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee1162a-ea93-4c08-959c-227fabca6e72-kube-api-access-j4jrg" (OuterVolumeSpecName: "kube-api-access-j4jrg") pod "aee1162a-ea93-4c08-959c-227fabca6e72" (UID: "aee1162a-ea93-4c08-959c-227fabca6e72"). InnerVolumeSpecName "kube-api-access-j4jrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:40:14.720294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.720272    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-util" (OuterVolumeSpecName: "util") pod "aee1162a-ea93-4c08-959c-227fabca6e72" (UID: "aee1162a-ea93-4c08-959c-227fabca6e72"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:40:14.815860 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.815833    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4jrg\" (UniqueName: \"kubernetes.io/projected/aee1162a-ea93-4c08-959c-227fabca6e72-kube-api-access-j4jrg\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:40:14.815860 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.815856    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-bundle\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:40:14.815860 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:14.815865    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aee1162a-ea93-4c08-959c-227fabca6e72-util\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:40:15.529154 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:15.529122    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t" event={"ID":"aee1162a-ea93-4c08-959c-227fabca6e72","Type":"ContainerDied","Data":"ca8954461143c55134cbfbd57ec217f544a3499351a65afaab9f0980ebac1000"}
Jul 18 13:40:15.529154 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:15.529155    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvlf5t"
Jul 18 13:40:15.529589 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:15.529169    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca8954461143c55134cbfbd57ec217f544a3499351a65afaab9f0980ebac1000"
Jul 18 13:40:18.488610 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:18.488570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:18.588065 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:18.588017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:25.358770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:25.358736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:25.460179 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:25.460143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:26.518767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:26.518734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:26.619527 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:26.619498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:27.365775 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:27.365742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:35.580966 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.580904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:35.644012 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.643980    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"]
Jul 18 13:40:35.644276 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644262    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="pull"
Jul 18 13:40:35.644276 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644276    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="pull"
Jul 18 13:40:35.644374 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644291    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="extract"
Jul 18 13:40:35.644374 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644297    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="extract"
Jul 18 13:40:35.644374 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644307    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="util"
Jul 18 13:40:35.644374 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644312    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="util"
Jul 18 13:40:35.644374 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.644364    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee1162a-ea93-4c08-959c-227fabca6e72" containerName="extract"
Jul 18 13:40:35.646172 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.646149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.649207 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.649181    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 18 13:40:35.649207 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.649199    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 18 13:40:35.649389 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.649224    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 18 13:40:35.649389 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.649300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 18 13:40:35.649389 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.649308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lp2dm\""
Jul 18 13:40:35.649575 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.649558    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 18 13:40:35.655767 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.655748    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"]
Jul 18 13:40:35.662920 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.662889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:35.663788 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.663768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dcab15d-73a0-4e64-aa78-4511f3b43358-cert\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.663890 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.663798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1dcab15d-73a0-4e64-aa78-4511f3b43358-manager-config\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.663890 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.663822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jzcr\" (UniqueName: \"kubernetes.io/projected/1dcab15d-73a0-4e64-aa78-4511f3b43358-kube-api-access-4jzcr\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.663969 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.663921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dcab15d-73a0-4e64-aa78-4511f3b43358-metrics-cert\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.764403 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.764372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jzcr\" (UniqueName: \"kubernetes.io/projected/1dcab15d-73a0-4e64-aa78-4511f3b43358-kube-api-access-4jzcr\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.764582 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.764446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dcab15d-73a0-4e64-aa78-4511f3b43358-metrics-cert\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.764582 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.764498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dcab15d-73a0-4e64-aa78-4511f3b43358-cert\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.764582 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.764521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1dcab15d-73a0-4e64-aa78-4511f3b43358-manager-config\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.765611 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.765584    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1dcab15d-73a0-4e64-aa78-4511f3b43358-manager-config\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.767574 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.767550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dcab15d-73a0-4e64-aa78-4511f3b43358-metrics-cert\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.767689 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.767668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dcab15d-73a0-4e64-aa78-4511f3b43358-cert\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.777262 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.777238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jzcr\" (UniqueName: \"kubernetes.io/projected/1dcab15d-73a0-4e64-aa78-4511f3b43358-kube-api-access-4jzcr\") pod \"lws-controller-manager-8664c48d9f-2r8tw\" (UID: \"1dcab15d-73a0-4e64-aa78-4511f3b43358\") " pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:35.955604 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:35.955526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:36.077510 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:36.077445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"]
Jul 18 13:40:36.079648 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:40:36.079618    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dcab15d_73a0_4e64_aa78_4511f3b43358.slice/crio-dc1b11e1d5aa2dd052b9532edf9a7fcdf83662fd5bdb9dad917c0a2b6037e842 WatchSource:0}: Error finding container dc1b11e1d5aa2dd052b9532edf9a7fcdf83662fd5bdb9dad917c0a2b6037e842: Status 404 returned error can't find the container with id dc1b11e1d5aa2dd052b9532edf9a7fcdf83662fd5bdb9dad917c0a2b6037e842
Jul 18 13:40:36.612143 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:36.612108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw" event={"ID":"1dcab15d-73a0-4e64-aa78-4511f3b43358","Type":"ContainerStarted","Data":"dc1b11e1d5aa2dd052b9532edf9a7fcdf83662fd5bdb9dad917c0a2b6037e842"}
Jul 18 13:40:38.621926 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:38.621888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw" event={"ID":"1dcab15d-73a0-4e64-aa78-4511f3b43358","Type":"ContainerStarted","Data":"1692dd097c782638fe475dbae0de93aeb2a0f1b42b57f56f671d71a7b720e561"}
Jul 18 13:40:38.622348 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:38.622018    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:38.641507 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:38.641458    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw" podStartSLOduration=1.8936941919999999 podStartE2EDuration="3.641443914s" totalImagesPullingTime="1.747749722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:40:38.639761679 +0000 UTC m=+488.296451806" watchObservedRunningTime="2026-07-18 13:40:38.641443914 +0000 UTC m=+488.298134043"
Jul 18 13:40:39.107838 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:39.107808    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:39.199534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:39.199487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:39.713371 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:39.713332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:39.952409 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:39.952371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.055239 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.055209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.107913 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.106173    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"]
Jul 18 13:40:40.109875 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.109852    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.112160 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.112131    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 13:40:40.112296 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.112254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zj28h\""
Jul 18 13:40:40.112296 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.112267    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 13:40:40.112427 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.112294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 13:40:40.112507 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.112491    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 13:40:40.117657 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.117636    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"]
Jul 18 13:40:40.198918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.198888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6575cad-578b-4d1a-a883-d27b724456f8-manifests\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.199103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.198934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vms97\" (UniqueName: \"kubernetes.io/projected/c6575cad-578b-4d1a-a883-d27b724456f8-kube-api-access-vms97\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.199103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.198974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6575cad-578b-4d1a-a883-d27b724456f8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.199103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.199000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6575cad-578b-4d1a-a883-d27b724456f8-tmp\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.199103 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.199020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6575cad-578b-4d1a-a883-d27b724456f8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.203429 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.203393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.300261 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.300229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6575cad-578b-4d1a-a883-d27b724456f8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.300261 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.300266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6575cad-578b-4d1a-a883-d27b724456f8-tmp\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.300465 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.300291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6575cad-578b-4d1a-a883-d27b724456f8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.300465 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.300347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6575cad-578b-4d1a-a883-d27b724456f8-manifests\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.300465 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.300388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vms97\" (UniqueName: \"kubernetes.io/projected/c6575cad-578b-4d1a-a883-d27b724456f8-kube-api-access-vms97\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.300788 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.300767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6575cad-578b-4d1a-a883-d27b724456f8-manifests\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.302677 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.302654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6575cad-578b-4d1a-a883-d27b724456f8-tmp\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.302773 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.302699    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6575cad-578b-4d1a-a883-d27b724456f8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.302842 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.302825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6575cad-578b-4d1a-a883-d27b724456f8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.304092 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.304066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.307863 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.307818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vms97\" (UniqueName: \"kubernetes.io/projected/c6575cad-578b-4d1a-a883-d27b724456f8-kube-api-access-vms97\") pod \"opendatahub-operator-controller-manager-7bd7687b4d-mdsp9\" (UID: \"c6575cad-578b-4d1a-a883-d27b724456f8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.410179 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.410148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.421938 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.421916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:40.516273 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.516244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.568232 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.568162    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"]
Jul 18 13:40:40.571517 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:40:40.571488    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6575cad_578b_4d1a_a883_d27b724456f8.slice/crio-acb893f61e7ca6cfa7d6aa1854afa9b26ef9368636227fb8771a044dd2794c15 WatchSource:0}: Error finding container acb893f61e7ca6cfa7d6aa1854afa9b26ef9368636227fb8771a044dd2794c15: Status 404 returned error can't find the container with id acb893f61e7ca6cfa7d6aa1854afa9b26ef9368636227fb8771a044dd2794c15
Jul 18 13:40:40.606332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.606297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:40.631032 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:40.631002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9" event={"ID":"c6575cad-578b-4d1a-a883-d27b724456f8","Type":"ContainerStarted","Data":"acb893f61e7ca6cfa7d6aa1854afa9b26ef9368636227fb8771a044dd2794c15"}
Jul 18 13:40:43.648141 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:43.648100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9" event={"ID":"c6575cad-578b-4d1a-a883-d27b724456f8","Type":"ContainerStarted","Data":"7331d5e9ffd2c42b39104c340f21cb9c25b356e2a160f47869ba9073077ac644"}
Jul 18 13:40:44.653496 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:44.653449    2577 generic.go:356] "Generic (PLEG): container finished" podID="c6575cad-578b-4d1a-a883-d27b724456f8" containerID="7331d5e9ffd2c42b39104c340f21cb9c25b356e2a160f47869ba9073077ac644" exitCode=0
Jul 18 13:40:44.653890 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:44.653545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9" event={"ID":"c6575cad-578b-4d1a-a883-d27b724456f8","Type":"ContainerDied","Data":"7331d5e9ffd2c42b39104c340f21cb9c25b356e2a160f47869ba9073077ac644"}
Jul 18 13:40:45.660266 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:45.660230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9" event={"ID":"c6575cad-578b-4d1a-a883-d27b724456f8","Type":"ContainerStarted","Data":"342acdabf358c9520f173a7b49b1f5facbd77eb69f9ad7ca52e9d04de67eb733"}
Jul 18 13:40:45.660634 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:45.660315    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:45.681877 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:45.681828    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9" podStartSLOduration=1.465743671 podStartE2EDuration="5.681811243s" totalImagesPullingTime="3.202842224s" totalInitContainerRuntime="1.013225348s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:40:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:40:45.679298929 +0000 UTC m=+495.335989082" watchObservedRunningTime="2026-07-18 13:40:45.681811243 +0000 UTC m=+495.338501371"
Jul 18 13:40:49.628892 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:49.628863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8664c48d9f-2r8tw"
Jul 18 13:40:50.064486 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:50.064445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:56.668192 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:56.668155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd7687b4d-mdsp9"
Jul 18 13:40:57.106740 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:57.106699    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:57.207301 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:57.207264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:58.373602 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:58.373570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:58.535858 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:58.535822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:59.127823 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:59.127780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:40:59.236304 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:40:59.236263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:04.078351 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:04.078318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:04.194729 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:04.194693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:04.355811 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:04.355724    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:04.417228 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:04.417194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:04.516422 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:04.516382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:04.803986 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:04.803941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:05.244668 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:05.244588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:05.313898 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:05.313859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:06.219968 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:06.219930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:10.940649 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:10.940611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:11.033813 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.033778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:11.563605 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.563572    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"]
Jul 18 13:41:11.568938 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.568918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.571324 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.571299    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 18 13:41:11.571682 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.571659    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mlgfz\""
Jul 18 13:41:11.580391 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.580367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"]
Jul 18 13:41:11.594547 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.593465    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"]
Jul 18 13:41:11.600664 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.600639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.602824 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.602802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 18 13:41:11.603130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.603103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 18 13:41:11.603224 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.603120    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 18 13:41:11.603224 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.603210    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gpjhj\""
Jul 18 13:41:11.606115 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.606093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"]
Jul 18 13:41:11.608687 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.608560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:11.675564 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.675564 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfwdx\" (UniqueName: \"kubernetes.io/projected/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-kube-api-access-vfwdx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.675824 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675649    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.675824 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.675824 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.675824 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.676004 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.676004 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.676004 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.675887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.776553 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.776553 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lppc5\" (UniqueName: \"kubernetes.io/projected/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-kube-api-access-lppc5\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.776873 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.776991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.777104    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.777123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.777336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfwdx\" (UniqueName: \"kubernetes.io/projected/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-kube-api-access-vfwdx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.777405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.777534 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.777497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.778025 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.777752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.786497 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.779553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.787059 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.787017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.796638 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.796609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.807120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.807091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfwdx\" (UniqueName: \"kubernetes.io/projected/55be83ab-4c95-4a71-9c08-b6a9ead5eea1-kube-api-access-vfwdx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jr5xd\" (UID: \"55be83ab-4c95-4a71-9c08-b6a9ead5eea1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.860276 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.860183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:11.878130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.878130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.878352 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.878352 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.878352 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lppc5\" (UniqueName: \"kubernetes.io/projected/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-kube-api-access-lppc5\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.878352 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.878352 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.878271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.879137 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.879103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.880589 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.880561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.880729 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.880707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.880796 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.880734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:11.880926 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.880904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.881003 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.880988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.891673 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.891651    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.894238 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.894215    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lppc5\" (UniqueName: \"kubernetes.io/projected/f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e-kube-api-access-lppc5\") pod \"istiod-openshift-gateway-94bb8fbfd-q8kj4\" (UID: \"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:11.913074 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:11.913045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:12.000332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:12.000227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:12.049336 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:12.049073    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"]
Jul 18 13:41:12.057674 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:41:12.057387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55be83ab_4c95_4a71_9c08_b6a9ead5eea1.slice/crio-ec6e3d16bc7fb253244a96d3d1b29c8301a1e95fc22dcf208c96c3384b1f9b27 WatchSource:0}: Error finding container ec6e3d16bc7fb253244a96d3d1b29c8301a1e95fc22dcf208c96c3384b1f9b27: Status 404 returned error can't find the container with id ec6e3d16bc7fb253244a96d3d1b29c8301a1e95fc22dcf208c96c3384b1f9b27
Jul 18 13:41:12.104469 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:12.104433    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"]
Jul 18 13:41:12.789971 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:12.789897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4" event={"ID":"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e","Type":"ContainerStarted","Data":"a0027cd28d6fb42221924a618403a6bab458a7b9a2ba2f64b7d56174ffe6442f"}
Jul 18 13:41:12.792102 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:12.792012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd" event={"ID":"55be83ab-4c95-4a71-9c08-b6a9ead5eea1","Type":"ContainerStarted","Data":"ec6e3d16bc7fb253244a96d3d1b29c8301a1e95fc22dcf208c96c3384b1f9b27"}
Jul 18 13:41:14.161293 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:14.161197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:14.284615 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:14.284575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:14.430493 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:14.430336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:14.721728 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:14.721642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:14.825725 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:14.825639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:15.476283 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.476244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:15.577233 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.577194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:15.631147 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.631094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:15.821050 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.820996    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 13:41:15.821136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.821096    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 13:41:15.825707 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.825672    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 13:41:15.825809 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.825734    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 13:41:15.825809 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:15.825762    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 13:41:16.814000 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.813959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4" event={"ID":"f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e","Type":"ContainerStarted","Data":"faed718d117309bf752fcab8457ded152a09560e1fe2d4431dcdb416a0c0feb7"}
Jul 18 13:41:16.814487 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.814169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:16.815758 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.815731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd" event={"ID":"55be83ab-4c95-4a71-9c08-b6a9ead5eea1","Type":"ContainerStarted","Data":"1571aab2365eef8288d20c7e924532f23f0f05daf659fad045dbbc5f6b1f9462"}
Jul 18 13:41:16.815880 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.815861    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-q8kj4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 13:41:16.815954 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.815922    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4" podUID="f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 13:41:16.833770 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.833725    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4" podStartSLOduration=2.130580303 podStartE2EDuration="5.833710554s" totalImagesPullingTime="3.703130251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:41:16.832592202 +0000 UTC m=+526.489282333" watchObservedRunningTime="2026-07-18 13:41:16.833710554 +0000 UTC m=+526.490400683"
Jul 18 13:41:16.854307 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.854231    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd" podStartSLOduration=2.089785067 podStartE2EDuration="5.854202763s" totalImagesPullingTime="3.764417696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:41:16.851499909 +0000 UTC m=+526.508190037" watchObservedRunningTime="2026-07-18 13:41:16.854202763 +0000 UTC m=+526.510892893"
Jul 18 13:41:16.881130 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.881089    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:16.887153 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:16.887125    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:17.820994 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:17.820957    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:17.821432 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:17.821335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-q8kj4"
Jul 18 13:41:17.821865 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:17.821847    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jr5xd"
Jul 18 13:41:51.188766 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:51.188713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:52.940715 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:52.940681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:59.856577 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:59.856542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:41:59.959916 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:41:59.959879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:00.054172 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:00.054140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:00.409476 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:00.409437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:00.509991 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:00.509957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:00.779558 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:00.779525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:01.101194 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:01.101104    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:21.614926 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:21.614870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:30.317317 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:30.317284    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:30.723087 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:30.722999    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:42:30.724019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:30.723991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:42:30.729732 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:30.729713    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:42:30.730715 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:30.730701    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:42:31.709002 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:31.708963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:31.756631 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:31.756600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:33.704918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:33.704881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:34.316538 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:34.316505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:35.741947 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:35.741900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:37.742508 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:37.742463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:38.105307 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:38.105272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:40.369486 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:40.369439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:40.922302 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:40.922267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:42.143598 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:42.143552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:42.246228 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:42.246191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:42.346748 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:42.346707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:42.445741 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:42.445655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:43.375674 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:43.375635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:46.124526 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:46.124488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:46.225406 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:46.225371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:51.513375 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:51.513342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:51.618734 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:51.618690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:51.941034 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:51.940998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:52.046505 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:52.046472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:52.368819 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:52.368784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:52.513976 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:52.513897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:55.710380 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:55.710344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:55.810540 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:55.810482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:57.839690 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:57.839648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:42:57.939924 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:42:57.939891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:04.496177 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:04.496143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:04.602465 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:04.602427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:09.871279 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:09.871236    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:09.921025 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:09.920987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:10.474136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:10.474099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:10.526634 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:10.526600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:14.277182 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:14.277141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:14.380094 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:14.380032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:17.476064 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:17.476016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:17.576690 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:17.576658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:34.731838 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:34.731793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:34.904443 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:34.904405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:49.077395 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:49.077361    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:49.186946 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:49.186907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:49.239002 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:49.238963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:49.477995 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:49.477914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:49.537101 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:49.537067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:49.913419 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:49.913390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:43:50.014032 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:43:50.013997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:07.000705 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.000637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:07.095865 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.095815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:07.455115 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.455081    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"]
Jul 18 13:44:07.458231 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.458209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.460772 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.460741    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 18 13:44:07.460898 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.460874    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dnfqg\""
Jul 18 13:44:07.462112 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.462091    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 13:44:07.462211 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.462116    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 13:44:07.476638 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.476617    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"]
Jul 18 13:44:07.499141 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.499103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:07.553074 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.553020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ls4w\" (UniqueName: \"kubernetes.io/projected/983920a3-8a28-428a-b439-91f3603926d9-kube-api-access-6ls4w\") pod \"postgres-69fb4594cf-8nbxd\" (UID: \"983920a3-8a28-428a-b439-91f3603926d9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.553271 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.553089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/983920a3-8a28-428a-b439-91f3603926d9-data\") pod \"postgres-69fb4594cf-8nbxd\" (UID: \"983920a3-8a28-428a-b439-91f3603926d9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.653615 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.653570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/983920a3-8a28-428a-b439-91f3603926d9-data\") pod \"postgres-69fb4594cf-8nbxd\" (UID: \"983920a3-8a28-428a-b439-91f3603926d9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.653786 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.653683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ls4w\" (UniqueName: \"kubernetes.io/projected/983920a3-8a28-428a-b439-91f3603926d9-kube-api-access-6ls4w\") pod \"postgres-69fb4594cf-8nbxd\" (UID: \"983920a3-8a28-428a-b439-91f3603926d9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.653986 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.653966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/983920a3-8a28-428a-b439-91f3603926d9-data\") pod \"postgres-69fb4594cf-8nbxd\" (UID: \"983920a3-8a28-428a-b439-91f3603926d9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.661626 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.661603    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ls4w\" (UniqueName: \"kubernetes.io/projected/983920a3-8a28-428a-b439-91f3603926d9-kube-api-access-6ls4w\") pod \"postgres-69fb4594cf-8nbxd\" (UID: \"983920a3-8a28-428a-b439-91f3603926d9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.768744 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.768663    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:07.900721 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:07.900691    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"]
Jul 18 13:44:07.902614 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:44:07.902589    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983920a3_8a28_428a_b439_91f3603926d9.slice/crio-64678f662af8b1a50d5ffd43d1e8a533d9589f9ddc0cd6ca5daa5fb36668f62f WatchSource:0}: Error finding container 64678f662af8b1a50d5ffd43d1e8a533d9589f9ddc0cd6ca5daa5fb36668f62f: Status 404 returned error can't find the container with id 64678f662af8b1a50d5ffd43d1e8a533d9589f9ddc0cd6ca5daa5fb36668f62f
Jul 18 13:44:08.560214 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:08.560178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd" event={"ID":"983920a3-8a28-428a-b439-91f3603926d9","Type":"ContainerStarted","Data":"64678f662af8b1a50d5ffd43d1e8a533d9589f9ddc0cd6ca5daa5fb36668f62f"}
Jul 18 13:44:13.586516 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:13.586483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd" event={"ID":"983920a3-8a28-428a-b439-91f3603926d9","Type":"ContainerStarted","Data":"e02560d57a98bc4d39fb89bc97fb639ef6c89569039a8ad4ff5d72ba8f9c6d99"}
Jul 18 13:44:13.586897 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:13.586650    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:13.602508 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:13.602453    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd" podStartSLOduration=1.808705461 podStartE2EDuration="6.602434907s" totalImagesPullingTime="4.793729446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:44:13.59982992 +0000 UTC m=+703.256520050" watchObservedRunningTime="2026-07-18 13:44:13.602434907 +0000 UTC m=+703.259125039"
Jul 18 13:44:19.621150 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:19.621123    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8nbxd"
Jul 18 13:44:19.905589 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:19.905501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:20.128197 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:20.128168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:26.169461 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.169375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:26.184931 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.184896    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67bf744f4-dhg7p"]
Jul 18 13:44:26.188406 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.188385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.190633 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.190612    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xbm2b\""
Jul 18 13:44:26.190769 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.190648    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 13:44:26.190769 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.190663    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 13:44:26.201618 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.201595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bf744f4-dhg7p"]
Jul 18 13:44:26.269582 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.269545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:26.319945 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.319912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.320128 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.319981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqchc\" (UniqueName: \"kubernetes.io/projected/9c40ac34-21a3-40f8-a86f-7e8dc746022b-kube-api-access-mqchc\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.350651 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.350614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:26.420474 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.420372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqchc\" (UniqueName: \"kubernetes.io/projected/9c40ac34-21a3-40f8-a86f-7e8dc746022b-kube-api-access-mqchc\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.420637 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.420488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.420637 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:44:26.420614    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 18 13:44:26.420762 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:44:26.420700    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert podName:9c40ac34-21a3-40f8-a86f-7e8dc746022b nodeName:}" failed. No retries permitted until 2026-07-18 13:44:26.920678364 +0000 UTC m=+716.577368474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert") pod "maas-controller-67bf744f4-dhg7p" (UID: "9c40ac34-21a3-40f8-a86f-7e8dc746022b") : secret "maas-controller-webhook-cert" not found
Jul 18 13:44:26.431071 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.431050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqchc\" (UniqueName: \"kubernetes.io/projected/9c40ac34-21a3-40f8-a86f-7e8dc746022b-kube-api-access-mqchc\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.660411 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.660376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:26.925150 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.925105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:26.927569 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:26.927550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert\") pod \"maas-controller-67bf744f4-dhg7p\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") " pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:27.098999 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:27.098962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:27.228373 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:44:27.228338    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c40ac34_21a3_40f8_a86f_7e8dc746022b.slice/crio-182afd54f8c452f88bc1d55ed1cbcb5ed21c556115862f248c7037a2a6a91785 WatchSource:0}: Error finding container 182afd54f8c452f88bc1d55ed1cbcb5ed21c556115862f248c7037a2a6a91785: Status 404 returned error can't find the container with id 182afd54f8c452f88bc1d55ed1cbcb5ed21c556115862f248c7037a2a6a91785
Jul 18 13:44:27.232365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:27.232338    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bf744f4-dhg7p"]
Jul 18 13:44:27.647962 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:27.647929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bf744f4-dhg7p" event={"ID":"9c40ac34-21a3-40f8-a86f-7e8dc746022b","Type":"ContainerStarted","Data":"182afd54f8c452f88bc1d55ed1cbcb5ed21c556115862f248c7037a2a6a91785"}
Jul 18 13:44:29.615256 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:29.615198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:29.659651 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:29.659616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bf744f4-dhg7p" event={"ID":"9c40ac34-21a3-40f8-a86f-7e8dc746022b","Type":"ContainerStarted","Data":"391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14"}
Jul 18 13:44:29.659819 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:29.659679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:29.693012 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:29.692951    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67bf744f4-dhg7p" podStartSLOduration=1.455027331 podStartE2EDuration="3.692932325s" totalImagesPullingTime="2.237904994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:44:29.692256772 +0000 UTC m=+719.348946899" watchObservedRunningTime="2026-07-18 13:44:29.692932325 +0000 UTC m=+719.349622455"
Jul 18 13:44:29.714789 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:29.714756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:44:40.667276 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:40.667242    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:41.057727 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.057693    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67bf744f4-dhg7p"]
Jul 18 13:44:41.057889 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.057867    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67bf744f4-dhg7p" podUID="9c40ac34-21a3-40f8-a86f-7e8dc746022b" containerName="manager" containerID="cri-o://391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14" gracePeriod=30
Jul 18 13:44:41.305082 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.305057    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:41.448849 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.448764    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqchc\" (UniqueName: \"kubernetes.io/projected/9c40ac34-21a3-40f8-a86f-7e8dc746022b-kube-api-access-mqchc\") pod \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") "
Jul 18 13:44:41.448849 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.448814    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert\") pod \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\" (UID: \"9c40ac34-21a3-40f8-a86f-7e8dc746022b\") "
Jul 18 13:44:41.450932 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.450904    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c40ac34-21a3-40f8-a86f-7e8dc746022b-kube-api-access-mqchc" (OuterVolumeSpecName: "kube-api-access-mqchc") pod "9c40ac34-21a3-40f8-a86f-7e8dc746022b" (UID: "9c40ac34-21a3-40f8-a86f-7e8dc746022b"). InnerVolumeSpecName "kube-api-access-mqchc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:44:41.451062 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.450992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9c40ac34-21a3-40f8-a86f-7e8dc746022b" (UID: "9c40ac34-21a3-40f8-a86f-7e8dc746022b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:44:41.549730 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.549694    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqchc\" (UniqueName: \"kubernetes.io/projected/9c40ac34-21a3-40f8-a86f-7e8dc746022b-kube-api-access-mqchc\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:44:41.549730 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.549725    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c40ac34-21a3-40f8-a86f-7e8dc746022b-webhook-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:44:41.716155 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.716057    2577 generic.go:356] "Generic (PLEG): container finished" podID="9c40ac34-21a3-40f8-a86f-7e8dc746022b" containerID="391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14" exitCode=0
Jul 18 13:44:41.716155 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.716107    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bf744f4-dhg7p"
Jul 18 13:44:41.716155 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.716126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bf744f4-dhg7p" event={"ID":"9c40ac34-21a3-40f8-a86f-7e8dc746022b","Type":"ContainerDied","Data":"391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14"}
Jul 18 13:44:41.716648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.716179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bf744f4-dhg7p" event={"ID":"9c40ac34-21a3-40f8-a86f-7e8dc746022b","Type":"ContainerDied","Data":"182afd54f8c452f88bc1d55ed1cbcb5ed21c556115862f248c7037a2a6a91785"}
Jul 18 13:44:41.716648 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.716209    2577 scope.go:117] "RemoveContainer" containerID="391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14"
Jul 18 13:44:41.725067 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.725022    2577 scope.go:117] "RemoveContainer" containerID="391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14"
Jul 18 13:44:41.725311 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:44:41.725295    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14\": container with ID starting with 391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14 not found: ID does not exist" containerID="391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14"
Jul 18 13:44:41.725351 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.725319    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14"} err="failed to get container status \"391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14\": rpc error: code = NotFound desc = could not find container \"391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14\": container with ID starting with 391c413bf4277b4ca2d1d4709a7111646fc0828ad91f605a1441ed68c65dcc14 not found: ID does not exist"
Jul 18 13:44:41.737748 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.737718    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67bf744f4-dhg7p"]
Jul 18 13:44:41.739363 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:41.739344    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67bf744f4-dhg7p"]
Jul 18 13:44:42.802832 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:44:42.802799    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c40ac34-21a3-40f8-a86f-7e8dc746022b" path="/var/lib/kubelet/pods/9c40ac34-21a3-40f8-a86f-7e8dc746022b/volumes"
Jul 18 13:45:00.032713 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:00.032608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:00.132720 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:00.132664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:00.544669 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:00.544636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:00.693988 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:00.693954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:00.998346 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:00.998262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:01.082257 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:01.082219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:11.169592 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:11.169559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:11.280813 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:11.280777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:19.880898 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:19.880853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:33.618178 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:33.618123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:43.478005 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.477971    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"]
Jul 18 13:45:43.478478 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.478459    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c40ac34-21a3-40f8-a86f-7e8dc746022b" containerName="manager"
Jul 18 13:45:43.478554 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.478480    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c40ac34-21a3-40f8-a86f-7e8dc746022b" containerName="manager"
Jul 18 13:45:43.478614 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.478574    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c40ac34-21a3-40f8-a86f-7e8dc746022b" containerName="manager"
Jul 18 13:45:43.483312 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.483288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.486430 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.486347    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 13:45:43.486430 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.486383    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 18 13:45:43.486618 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.486454    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6l8xf\""
Jul 18 13:45:43.486618 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.486352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 13:45:43.492010 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.491986    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"]
Jul 18 13:45:43.518302 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.518266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:43.560120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.560083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.560120 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.560123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knl8k\" (UniqueName: \"kubernetes.io/projected/8aca100b-507f-4428-94ce-2b294ef5862d-kube-api-access-knl8k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.560294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.560154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.560294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.560174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aca100b-507f-4428-94ce-2b294ef5862d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.560294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.560218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.560294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.560234    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661355 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aca100b-507f-4428-94ce-2b294ef5862d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661620 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661620 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661620 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661620 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knl8k\" (UniqueName: \"kubernetes.io/projected/8aca100b-507f-4428-94ce-2b294ef5862d-kube-api-access-knl8k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661881 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661849    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661998 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.661998 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.661920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.663711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.663688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aca100b-507f-4428-94ce-2b294ef5862d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.663877 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.663863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aca100b-507f-4428-94ce-2b294ef5862d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.668230 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.668211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knl8k\" (UniqueName: \"kubernetes.io/projected/8aca100b-507f-4428-94ce-2b294ef5862d-kube-api-access-knl8k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwcq\" (UID: \"8aca100b-507f-4428-94ce-2b294ef5862d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.797551 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.797520    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:43.928481 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.928454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"]
Jul 18 13:45:43.930353 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:45:43.930321    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aca100b_507f_4428_94ce_2b294ef5862d.slice/crio-dd33d0ee5e9607dba184f2a3d26c6cdf380aae8a27f35a532112c4611b6bd212 WatchSource:0}: Error finding container dd33d0ee5e9607dba184f2a3d26c6cdf380aae8a27f35a532112c4611b6bd212: Status 404 returned error can't find the container with id dd33d0ee5e9607dba184f2a3d26c6cdf380aae8a27f35a532112c4611b6bd212
Jul 18 13:45:43.931607 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.931579    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 13:45:43.998894 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:43.998854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq" event={"ID":"8aca100b-507f-4428-94ce-2b294ef5862d","Type":"ContainerStarted","Data":"dd33d0ee5e9607dba184f2a3d26c6cdf380aae8a27f35a532112c4611b6bd212"}
Jul 18 13:45:45.005462 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:45.005425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq" event={"ID":"8aca100b-507f-4428-94ce-2b294ef5862d","Type":"ContainerStarted","Data":"8d9cea0a22b8e89284538e06f652e469e91b8cf8ff1049dda71946601c9712c3"}
Jul 18 13:45:45.022509 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:45.022450    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq" podStartSLOduration=1.049874166 podStartE2EDuration="2.022433449s" totalImagesPullingTime="972.559283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:45:45.021555786 +0000 UTC m=+794.678245924" watchObservedRunningTime="2026-07-18 13:45:45.022433449 +0000 UTC m=+794.679123577"
Jul 18 13:45:49.611997 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:49.611948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:53.797996 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:53.797917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:53.797996 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:53.797986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:53.810684 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:53.810658    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:54.063613 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:54.063524    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwcq"
Jul 18 13:45:55.588231 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.588197    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"]
Jul 18 13:45:55.591360 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.591342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.593591 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.593569    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 18 13:45:55.600703 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.600679    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"]
Jul 18 13:45:55.616388 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.616349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:45:55.664106 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.664071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.664106 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.664105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nq94\" (UniqueName: \"kubernetes.io/projected/57ec7d97-1938-489f-b361-c834d8e6dabc-kube-api-access-6nq94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.664347 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.664131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ec7d97-1938-489f-b361-c834d8e6dabc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.664347 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.664263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.664347 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.664302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.664472 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.664354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.765623 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.765576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.765623 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.765627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nq94\" (UniqueName: \"kubernetes.io/projected/57ec7d97-1938-489f-b361-c834d8e6dabc-kube-api-access-6nq94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.765842 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.765655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ec7d97-1938-489f-b361-c834d8e6dabc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.765842 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.765733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.765842 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.765758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.766058 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.766021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.766186 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.766053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.766186 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.766082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.766375 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.766355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.768461 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.768440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ec7d97-1938-489f-b361-c834d8e6dabc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.768639 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.768620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ec7d97-1938-489f-b361-c834d8e6dabc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.773795 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.773777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nq94\" (UniqueName: \"kubernetes.io/projected/57ec7d97-1938-489f-b361-c834d8e6dabc-kube-api-access-6nq94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n8dcr\" (UID: \"57ec7d97-1938-489f-b361-c834d8e6dabc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:55.901857 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:55.901775    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:45:56.032264 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:56.032229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"]
Jul 18 13:45:56.034612 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:45:56.034585    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ec7d97_1938_489f_b361_c834d8e6dabc.slice/crio-a55790a9d33be402a811839bf9db27fd31c1081b085fbc7af49e6e7d7315322e WatchSource:0}: Error finding container a55790a9d33be402a811839bf9db27fd31c1081b085fbc7af49e6e7d7315322e: Status 404 returned error can't find the container with id a55790a9d33be402a811839bf9db27fd31c1081b085fbc7af49e6e7d7315322e
Jul 18 13:45:56.059019 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:56.058988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr" event={"ID":"57ec7d97-1938-489f-b361-c834d8e6dabc","Type":"ContainerStarted","Data":"a55790a9d33be402a811839bf9db27fd31c1081b085fbc7af49e6e7d7315322e"}
Jul 18 13:45:57.065512 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:57.065478    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr" event={"ID":"57ec7d97-1938-489f-b361-c834d8e6dabc","Type":"ContainerStarted","Data":"5695f2a00c0356077f1d4035849555994a85e4b80320e2a5008482c770778d0f"}
Jul 18 13:45:57.081419 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:45:57.081374    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr" podStartSLOduration=1.9103289879999998 podStartE2EDuration="2.081358781s" totalImagesPullingTime="171.029793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:45:57.080160692 +0000 UTC m=+806.736850823" watchObservedRunningTime="2026-07-18 13:45:57.081358781 +0000 UTC m=+806.738048909"
Jul 18 13:46:05.902221 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:46:05.902186    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:46:05.902659 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:46:05.902244    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:46:05.914856 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:46:05.914828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:46:06.116565 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:46:06.116532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n8dcr"
Jul 18 13:46:08.501479 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:46:08.501445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:46:22.020636 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:46:22.020597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:47:04.657465 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:04.657428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:47:05.626433 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:05.626391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:47:05.760240 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:05.760182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:47:05.857587 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:05.857531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:47:30.748960 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:30.748933    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:47:30.751261 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:30.751233    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:47:30.757831 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:30.757812    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:47:30.765332 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:47:30.765313    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:48:06.222749 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:48:06.222698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:48:06.325821 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:48:06.325786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:48:20.328632 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:48:20.328561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:48:26.539525 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:48:26.539493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:48:51.679756 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:48:51.679720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:49:13.184668 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.184580    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f744f9d59-fjqxx"]
Jul 18 13:49:13.187948 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.187927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.191191 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.191172    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xbm2b\""
Jul 18 13:49:13.191917 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.191901    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 13:49:13.192000 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.191933    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 13:49:13.204622 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.204600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-fjqxx"]
Jul 18 13:49:13.332929 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.332909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf6r2\" (UniqueName: \"kubernetes.io/projected/3ed80609-30f3-4737-81f7-dce9915a1a35-kube-api-access-rf6r2\") pod \"maas-controller-5f744f9d59-fjqxx\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") " pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.333065 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.332940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed80609-30f3-4737-81f7-dce9915a1a35-webhook-cert\") pod \"maas-controller-5f744f9d59-fjqxx\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") " pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.433638 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.433614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf6r2\" (UniqueName: \"kubernetes.io/projected/3ed80609-30f3-4737-81f7-dce9915a1a35-kube-api-access-rf6r2\") pod \"maas-controller-5f744f9d59-fjqxx\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") " pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.433737 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.433654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed80609-30f3-4737-81f7-dce9915a1a35-webhook-cert\") pod \"maas-controller-5f744f9d59-fjqxx\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") " pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.435964 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.435917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed80609-30f3-4737-81f7-dce9915a1a35-webhook-cert\") pod \"maas-controller-5f744f9d59-fjqxx\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") " pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.441093 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.441068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf6r2\" (UniqueName: \"kubernetes.io/projected/3ed80609-30f3-4737-81f7-dce9915a1a35-kube-api-access-rf6r2\") pod \"maas-controller-5f744f9d59-fjqxx\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") " pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.498069 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.498030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:13.621945 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.621917    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-fjqxx"]
Jul 18 13:49:13.624144 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:49:13.624113    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed80609_30f3_4737_81f7_dce9915a1a35.slice/crio-a5607ea60c574df84ec7ec0758be4f9c91d95ad47173a1c1b81b649d7d577516 WatchSource:0}: Error finding container a5607ea60c574df84ec7ec0758be4f9c91d95ad47173a1c1b81b649d7d577516: Status 404 returned error can't find the container with id a5607ea60c574df84ec7ec0758be4f9c91d95ad47173a1c1b81b649d7d577516
Jul 18 13:49:13.942411 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:13.942377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f744f9d59-fjqxx" event={"ID":"3ed80609-30f3-4737-81f7-dce9915a1a35","Type":"ContainerStarted","Data":"a5607ea60c574df84ec7ec0758be4f9c91d95ad47173a1c1b81b649d7d577516"}
Jul 18 13:49:14.948645 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:14.948611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f744f9d59-fjqxx" event={"ID":"3ed80609-30f3-4737-81f7-dce9915a1a35","Type":"ContainerStarted","Data":"68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd"}
Jul 18 13:49:14.948997 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:14.948667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:14.965563 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:14.965516    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f744f9d59-fjqxx" podStartSLOduration=1.538292695 podStartE2EDuration="1.965498447s" totalImagesPullingTime="427.205752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:49:14.964034295 +0000 UTC m=+1004.620724422" watchObservedRunningTime="2026-07-18 13:49:14.965498447 +0000 UTC m=+1004.622188620"
Jul 18 13:49:23.352176 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:23.352142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:49:25.954824 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:25.954792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:30.448760 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.448725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:49:30.543003 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.542976    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-fjqxx"]
Jul 18 13:49:30.543224 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.543187    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f744f9d59-fjqxx" podUID="3ed80609-30f3-4737-81f7-dce9915a1a35" containerName="manager" containerID="cri-o://68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd" gracePeriod=30
Jul 18 13:49:30.788885 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.788861    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:30.869365 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.869340    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed80609-30f3-4737-81f7-dce9915a1a35-webhook-cert\") pod \"3ed80609-30f3-4737-81f7-dce9915a1a35\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") "
Jul 18 13:49:30.869515 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.869384    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf6r2\" (UniqueName: \"kubernetes.io/projected/3ed80609-30f3-4737-81f7-dce9915a1a35-kube-api-access-rf6r2\") pod \"3ed80609-30f3-4737-81f7-dce9915a1a35\" (UID: \"3ed80609-30f3-4737-81f7-dce9915a1a35\") "
Jul 18 13:49:30.871189 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.871162    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ed80609-30f3-4737-81f7-dce9915a1a35-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3ed80609-30f3-4737-81f7-dce9915a1a35" (UID: "3ed80609-30f3-4737-81f7-dce9915a1a35"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:49:30.871291 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.871269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed80609-30f3-4737-81f7-dce9915a1a35-kube-api-access-rf6r2" (OuterVolumeSpecName: "kube-api-access-rf6r2") pod "3ed80609-30f3-4737-81f7-dce9915a1a35" (UID: "3ed80609-30f3-4737-81f7-dce9915a1a35"). InnerVolumeSpecName "kube-api-access-rf6r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:49:30.970056 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.969966    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf6r2\" (UniqueName: \"kubernetes.io/projected/3ed80609-30f3-4737-81f7-dce9915a1a35-kube-api-access-rf6r2\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:49:30.970056 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:30.969994    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed80609-30f3-4737-81f7-dce9915a1a35-webhook-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:49:31.017961 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.017929    2577 generic.go:356] "Generic (PLEG): container finished" podID="3ed80609-30f3-4737-81f7-dce9915a1a35" containerID="68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd" exitCode=0
Jul 18 13:49:31.018098 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.017981    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f744f9d59-fjqxx"
Jul 18 13:49:31.018098 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.017997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f744f9d59-fjqxx" event={"ID":"3ed80609-30f3-4737-81f7-dce9915a1a35","Type":"ContainerDied","Data":"68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd"}
Jul 18 13:49:31.018098 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.018071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f744f9d59-fjqxx" event={"ID":"3ed80609-30f3-4737-81f7-dce9915a1a35","Type":"ContainerDied","Data":"a5607ea60c574df84ec7ec0758be4f9c91d95ad47173a1c1b81b649d7d577516"}
Jul 18 13:49:31.018234 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.018104    2577 scope.go:117] "RemoveContainer" containerID="68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd"
Jul 18 13:49:31.026322 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.026302    2577 scope.go:117] "RemoveContainer" containerID="68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd"
Jul 18 13:49:31.026549 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:49:31.026528    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd\": container with ID starting with 68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd not found: ID does not exist" containerID="68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd"
Jul 18 13:49:31.026610 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.026561    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd"} err="failed to get container status \"68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd\": rpc error: code = NotFound desc = could not find container \"68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd\": container with ID starting with 68c7a4c3bcf160c695eef36cc66bfd55a5329d185af54902dc4f6701500979dd not found: ID does not exist"
Jul 18 13:49:31.038016 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.037995    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-fjqxx"]
Jul 18 13:49:31.041420 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.041400    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-fjqxx"]
Jul 18 13:49:31.662132 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.662096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:49:31.771882 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.771852    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f744f9d59-qrv5k"]
Jul 18 13:49:31.772183 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.772172    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed80609-30f3-4737-81f7-dce9915a1a35" containerName="manager"
Jul 18 13:49:31.772230 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.772186    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed80609-30f3-4737-81f7-dce9915a1a35" containerName="manager"
Jul 18 13:49:31.772266 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.772261    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed80609-30f3-4737-81f7-dce9915a1a35" containerName="manager"
Jul 18 13:49:31.776528 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.776505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:31.779632 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.779613    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xbm2b\""
Jul 18 13:49:31.779632 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.779625    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 13:49:31.779781 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.779618    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 13:49:31.783105 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.783084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-qrv5k"]
Jul 18 13:49:31.877602 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.877575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77ac0aff-7a4c-4b52-8257-aa89371b6abd-webhook-cert\") pod \"maas-controller-5f744f9d59-qrv5k\" (UID: \"77ac0aff-7a4c-4b52-8257-aa89371b6abd\") " pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:31.877713 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.877650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr4b6\" (UniqueName: \"kubernetes.io/projected/77ac0aff-7a4c-4b52-8257-aa89371b6abd-kube-api-access-qr4b6\") pod \"maas-controller-5f744f9d59-qrv5k\" (UID: \"77ac0aff-7a4c-4b52-8257-aa89371b6abd\") " pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:31.978650 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.978588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr4b6\" (UniqueName: \"kubernetes.io/projected/77ac0aff-7a4c-4b52-8257-aa89371b6abd-kube-api-access-qr4b6\") pod \"maas-controller-5f744f9d59-qrv5k\" (UID: \"77ac0aff-7a4c-4b52-8257-aa89371b6abd\") " pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:31.978650 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.978626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77ac0aff-7a4c-4b52-8257-aa89371b6abd-webhook-cert\") pod \"maas-controller-5f744f9d59-qrv5k\" (UID: \"77ac0aff-7a4c-4b52-8257-aa89371b6abd\") " pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:31.980985 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.980958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77ac0aff-7a4c-4b52-8257-aa89371b6abd-webhook-cert\") pod \"maas-controller-5f744f9d59-qrv5k\" (UID: \"77ac0aff-7a4c-4b52-8257-aa89371b6abd\") " pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:31.991870 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:31.991843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr4b6\" (UniqueName: \"kubernetes.io/projected/77ac0aff-7a4c-4b52-8257-aa89371b6abd-kube-api-access-qr4b6\") pod \"maas-controller-5f744f9d59-qrv5k\" (UID: \"77ac0aff-7a4c-4b52-8257-aa89371b6abd\") " pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:32.088946 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:32.088923    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:32.212210 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:32.212187    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f744f9d59-qrv5k"]
Jul 18 13:49:32.213887 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:49:32.213862    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ac0aff_7a4c_4b52_8257_aa89371b6abd.slice/crio-77e9430d6752a107b44b77d1fe092aa9dda5dae5ee92ccb8ad955850505b7304 WatchSource:0}: Error finding container 77e9430d6752a107b44b77d1fe092aa9dda5dae5ee92ccb8ad955850505b7304: Status 404 returned error can't find the container with id 77e9430d6752a107b44b77d1fe092aa9dda5dae5ee92ccb8ad955850505b7304
Jul 18 13:49:32.802049 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:32.802018    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed80609-30f3-4737-81f7-dce9915a1a35" path="/var/lib/kubelet/pods/3ed80609-30f3-4737-81f7-dce9915a1a35/volumes"
Jul 18 13:49:33.029129 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:33.029096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f744f9d59-qrv5k" event={"ID":"77ac0aff-7a4c-4b52-8257-aa89371b6abd","Type":"ContainerStarted","Data":"062b25bb15a6333269c5011103aa238c28037bc49aae7f0b23b0c9bba4a725b4"}
Jul 18 13:49:33.029288 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:33.029173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f744f9d59-qrv5k" event={"ID":"77ac0aff-7a4c-4b52-8257-aa89371b6abd","Type":"ContainerStarted","Data":"77e9430d6752a107b44b77d1fe092aa9dda5dae5ee92ccb8ad955850505b7304"}
Jul 18 13:49:33.029288 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:33.029259    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:49:33.045936 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:33.045888    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f744f9d59-qrv5k" podStartSLOduration=1.7462223909999999 podStartE2EDuration="2.045875658s" totalImagesPullingTime="299.653267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:49:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:49:33.044228624 +0000 UTC m=+1022.700918755" watchObservedRunningTime="2026-07-18 13:49:33.045875658 +0000 UTC m=+1022.702565786"
Jul 18 13:49:44.035829 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:49:44.035794    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f744f9d59-qrv5k"
Jul 18 13:50:04.909085 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:04.909027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:05.009597 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:05.009558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:05.125477 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:05.125441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:10.947555 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:10.947517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:11.000013 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:10.999973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:11.066405 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:11.066370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:11.166749 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:11.166717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:11.271254 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:11.271179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:16.889465 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:16.889429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:50:17.006896 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:50:17.006861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:51:41.573709 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:51:41.573676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:52:30.783434 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:52:30.783403    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:52:30.786576 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:52:30.786554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:52:30.790033 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:52:30.790010    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:52:30.792814 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:52:30.792797    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:54:06.719449 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:54:06.719411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:55:20.581979 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:55:20.581901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:55:20.637306 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:55:20.637272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:55:24.479002 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:55:24.478950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:55:45.091987 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:55:45.091944    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:57:30.810666 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:57:30.810634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:57:30.815936 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:57:30.815917    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 13:57:30.817749 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:57:30.817729    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:57:30.822311 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:57:30.822288    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 13:58:04.524828 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:04.524739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:58:09.879847 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.879811    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"]
Jul 18 13:58:09.883594 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.883574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:09.885893 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.885868    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a9d318aa-kserve-self-signed-certs\""
Jul 18 13:58:09.893370 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.893344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"]
Jul 18 13:58:09.916674 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.916650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:58:09.916866 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.916850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830f3be2-78ce-498c-8b88-70445309859f-tls-certs\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:09.916918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.916878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-model-cache\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:09.916918 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.916903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-home\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:09.916997 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.916980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wnnc\" (UniqueName: \"kubernetes.io/projected/830f3be2-78ce-498c-8b88-70445309859f-kube-api-access-2wnnc\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:09.917066 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.917026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-tmp-dir\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:09.917110 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:09.917087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-dshm\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.018443 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830f3be2-78ce-498c-8b88-70445309859f-tls-certs\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.018711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-model-cache\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.018711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-home\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.018711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wnnc\" (UniqueName: \"kubernetes.io/projected/830f3be2-78ce-498c-8b88-70445309859f-kube-api-access-2wnnc\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.018711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-tmp-dir\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.018711 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-dshm\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.019022 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-model-cache\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.019022 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-home\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.019022 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.018975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-tmp-dir\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.020994 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.020971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-dshm\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.021282 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.021260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830f3be2-78ce-498c-8b88-70445309859f-tls-certs\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.026148 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.026126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wnnc\" (UniqueName: \"kubernetes.io/projected/830f3be2-78ce-498c-8b88-70445309859f-kube-api-access-2wnnc\") pod \"e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") " pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.194180 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.194097    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:10.322215 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.322183    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"]
Jul 18 13:58:10.323837 ip-10-0-128-123 kubenswrapper[2577]: W0718 13:58:10.323803    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod830f3be2_78ce_498c_8b88_70445309859f.slice/crio-76aa2a34103b83b11b72c441d186c818529da750f75f2f0a5bcdead77a9904b4 WatchSource:0}: Error finding container 76aa2a34103b83b11b72c441d186c818529da750f75f2f0a5bcdead77a9904b4: Status 404 returned error can't find the container with id 76aa2a34103b83b11b72c441d186c818529da750f75f2f0a5bcdead77a9904b4
Jul 18 13:58:10.424294 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.424254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s" event={"ID":"830f3be2-78ce-498c-8b88-70445309859f","Type":"ContainerStarted","Data":"d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa"}
Jul 18 13:58:10.424426 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.424305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s" event={"ID":"830f3be2-78ce-498c-8b88-70445309859f","Type":"ContainerStarted","Data":"76aa2a34103b83b11b72c441d186c818529da750f75f2f0a5bcdead77a9904b4"}
Jul 18 13:58:10.441823 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:10.441781    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s" podStartSLOduration=1.441765442 podStartE2EDuration="1.441765442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 13:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 13:58:10.439710876 +0000 UTC m=+1540.096401000" watchObservedRunningTime="2026-07-18 13:58:10.441765442 +0000 UTC m=+1540.098455571"
Jul 18 13:58:20.195251 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:20.195201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:20.195251 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:20.195260    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:20.207650 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:20.207622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:20.479814 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:20.479739    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:38.935458 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:38.935425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:58:40.506163 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:40.506120    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"]
Jul 18 13:58:40.527136 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:40.527102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 13:58:55.526967 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.526918    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s" podUID="830f3be2-78ce-498c-8b88-70445309859f" containerName="main" containerID="cri-o://d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa" gracePeriod=45
Jul 18 13:58:55.779554 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.779503    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:55.876572 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876537    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-tmp-dir\") pod \"830f3be2-78ce-498c-8b88-70445309859f\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") "
Jul 18 13:58:55.876715 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876580    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-dshm\") pod \"830f3be2-78ce-498c-8b88-70445309859f\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") "
Jul 18 13:58:55.876715 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876615    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-model-cache\") pod \"830f3be2-78ce-498c-8b88-70445309859f\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") "
Jul 18 13:58:55.876715 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876669    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-home\") pod \"830f3be2-78ce-498c-8b88-70445309859f\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") "
Jul 18 13:58:55.876881 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876720    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wnnc\" (UniqueName: \"kubernetes.io/projected/830f3be2-78ce-498c-8b88-70445309859f-kube-api-access-2wnnc\") pod \"830f3be2-78ce-498c-8b88-70445309859f\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") "
Jul 18 13:58:55.876881 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830f3be2-78ce-498c-8b88-70445309859f-tls-certs\") pod \"830f3be2-78ce-498c-8b88-70445309859f\" (UID: \"830f3be2-78ce-498c-8b88-70445309859f\") "
Jul 18 13:58:55.876881 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "830f3be2-78ce-498c-8b88-70445309859f" (UID: "830f3be2-78ce-498c-8b88-70445309859f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:55.877014 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876886    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-model-cache" (OuterVolumeSpecName: "model-cache") pod "830f3be2-78ce-498c-8b88-70445309859f" (UID: "830f3be2-78ce-498c-8b88-70445309859f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:55.877014 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.876962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-home" (OuterVolumeSpecName: "home") pod "830f3be2-78ce-498c-8b88-70445309859f" (UID: "830f3be2-78ce-498c-8b88-70445309859f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:55.877113 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.877091    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-model-cache\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:58:55.877113 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.877108    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-home\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:58:55.877191 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.877121    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-tmp-dir\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:58:55.878815 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.878796    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/830f3be2-78ce-498c-8b88-70445309859f-kube-api-access-2wnnc" (OuterVolumeSpecName: "kube-api-access-2wnnc") pod "830f3be2-78ce-498c-8b88-70445309859f" (UID: "830f3be2-78ce-498c-8b88-70445309859f"). InnerVolumeSpecName "kube-api-access-2wnnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 13:58:55.879211 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.879181    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-dshm" (OuterVolumeSpecName: "dshm") pod "830f3be2-78ce-498c-8b88-70445309859f" (UID: "830f3be2-78ce-498c-8b88-70445309859f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 13:58:55.879211 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.879191    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/830f3be2-78ce-498c-8b88-70445309859f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "830f3be2-78ce-498c-8b88-70445309859f" (UID: "830f3be2-78ce-498c-8b88-70445309859f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 13:58:55.978252 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.978220    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wnnc\" (UniqueName: \"kubernetes.io/projected/830f3be2-78ce-498c-8b88-70445309859f-kube-api-access-2wnnc\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:58:55.978252 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.978250    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830f3be2-78ce-498c-8b88-70445309859f-tls-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:58:55.978252 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:55.978261    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830f3be2-78ce-498c-8b88-70445309859f-dshm\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 13:58:56.637126 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.637086    2577 generic.go:356] "Generic (PLEG): container finished" podID="830f3be2-78ce-498c-8b88-70445309859f" containerID="d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa" exitCode=0
Jul 18 13:58:56.637556 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.637134    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"
Jul 18 13:58:56.637556 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.637166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s" event={"ID":"830f3be2-78ce-498c-8b88-70445309859f","Type":"ContainerDied","Data":"d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa"}
Jul 18 13:58:56.637556 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.637208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s" event={"ID":"830f3be2-78ce-498c-8b88-70445309859f","Type":"ContainerDied","Data":"76aa2a34103b83b11b72c441d186c818529da750f75f2f0a5bcdead77a9904b4"}
Jul 18 13:58:56.637556 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.637225    2577 scope.go:117] "RemoveContainer" containerID="d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa"
Jul 18 13:58:56.646264 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.646246    2577 scope.go:117] "RemoveContainer" containerID="d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa"
Jul 18 13:58:56.646541 ip-10-0-128-123 kubenswrapper[2577]: E0718 13:58:56.646522    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa\": container with ID starting with d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa not found: ID does not exist" containerID="d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa"
Jul 18 13:58:56.646585 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.646551    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa"} err="failed to get container status \"d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa\": rpc error: code = NotFound desc = could not find container \"d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa\": container with ID starting with d3677f6246f8b2d166590413057773bf70a5332f235f16279bb331a2bd26b2fa not found: ID does not exist"
Jul 18 13:58:56.657601 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.657578    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"]
Jul 18 13:58:56.660719 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.660698    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a9d318aa-kserve-864f594b55-ppv9s"]
Jul 18 13:58:56.806618 ip-10-0-128-123 kubenswrapper[2577]: I0718 13:58:56.806587    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830f3be2-78ce-498c-8b88-70445309859f" path="/var/lib/kubelet/pods/830f3be2-78ce-498c-8b88-70445309859f/volumes"
Jul 18 14:01:52.433153 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:52.433123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:53.546359 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:53.546325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:58.402352 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.402317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:58.502745 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.502714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:58.635101 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.635068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:58.912715 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.912684    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"]
Jul 18 14:01:58.913092 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.913075    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830f3be2-78ce-498c-8b88-70445309859f" containerName="main"
Jul 18 14:01:58.913092 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.913092    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="830f3be2-78ce-498c-8b88-70445309859f" containerName="main"
Jul 18 14:01:58.913227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.913200    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="830f3be2-78ce-498c-8b88-70445309859f" containerName="main"
Jul 18 14:01:58.916237 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.916221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.918623 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.918602    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7d23fa97-openshift-default-dockercfg-5dqmb\""
Jul 18 14:01:58.939258 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.939235    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"]
Jul 18 14:01:58.941958 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.941937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:58.974807 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.974782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istiod-ca-cert\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.974958 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.974824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln9lr\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-kube-api-access-ln9lr\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.974958 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.974859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-credential-socket\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.974958 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.974891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-certs\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.974958 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.974953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-token\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.975174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.974983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-socket\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.975174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.975067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-envoy\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.975174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.975099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-data\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:58.975174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:58.975147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-podinfo\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.044079 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.044022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:01:59.075815 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.075780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istiod-ca-cert\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.075961 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.075822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln9lr\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-kube-api-access-ln9lr\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076065 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-credential-socket\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076132 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-certs\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076189 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-token\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076189 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-socket\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076288 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-envoy\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076288 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-data\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076383 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-podinfo\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076439 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-credential-socket\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076511 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-certs\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076671 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istiod-ca-cert\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076734 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-data\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.076834 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.076817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-socket\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.078416 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.078375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-envoy\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.078640 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.078623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-podinfo\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.083311 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.083289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-token\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.083490 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.083467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln9lr\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-kube-api-access-ln9lr\") pod \"e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") " pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.226989 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.226912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:01:59.361625 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.361599    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"]
Jul 18 14:01:59.363429 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:01:59.363403    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod208dc340_b6f6_48d5_90aa_ccaedebfd4de.slice/crio-0dab1206025daca4ebf9bb59c821a0b20f1a1cd6b13fe0dbbbf5afe31553be3e WatchSource:0}: Error finding container 0dab1206025daca4ebf9bb59c821a0b20f1a1cd6b13fe0dbbbf5afe31553be3e: Status 404 returned error can't find the container with id 0dab1206025daca4ebf9bb59c821a0b20f1a1cd6b13fe0dbbbf5afe31553be3e
Jul 18 14:01:59.365540 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.365507    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:01:59.365636 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.365571    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:01:59.365636 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.365599    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:01:59.472602 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.472564    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" event={"ID":"208dc340-b6f6-48d5-90aa-ccaedebfd4de","Type":"ContainerStarted","Data":"6e2580acba62a45f72f995165f1372cf9897bd0742ecf4c73a054337a594c786"}
Jul 18 14:01:59.473065 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.472622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" event={"ID":"208dc340-b6f6-48d5-90aa-ccaedebfd4de","Type":"ContainerStarted","Data":"0dab1206025daca4ebf9bb59c821a0b20f1a1cd6b13fe0dbbbf5afe31553be3e"}
Jul 18 14:01:59.491576 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:01:59.491489    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podStartSLOduration=1.491474995 podStartE2EDuration="1.491474995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:01:59.49078155 +0000 UTC m=+1769.147471677" watchObservedRunningTime="2026-07-18 14:01:59.491474995 +0000 UTC m=+1769.148165123"
Jul 18 14:02:00.227053 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.227007    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:00.231770 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.231742    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:00.477489 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.477417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:00.478290 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.478270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:00.828890 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.828853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:00.902903 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.902864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:00.974062 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:00.974005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:01.175172 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:01.175067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:01.457328 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:01.457242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:01.556404 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:01.556360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:15.479183 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.479114    2577 patch_prober.go:28] interesting pod/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 18 14:02:15.479183 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.479173    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 18 14:02:15.546969 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.546939    2577 generic.go:356] "Generic (PLEG): container finished" podID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerID="6e2580acba62a45f72f995165f1372cf9897bd0742ecf4c73a054337a594c786" exitCode=0
Jul 18 14:02:15.547116 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.547008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" event={"ID":"208dc340-b6f6-48d5-90aa-ccaedebfd4de","Type":"ContainerDied","Data":"6e2580acba62a45f72f995165f1372cf9897bd0742ecf4c73a054337a594c786"}
Jul 18 14:02:15.547432 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.547416    2577 scope.go:117] "RemoveContainer" containerID="6e2580acba62a45f72f995165f1372cf9897bd0742ecf4c73a054337a594c786"
Jul 18 14:02:15.548372 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.548345    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:02:15.548438 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.548425    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:02:15.548474 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:15.548465    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:02:16.553640 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:16.553607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" event={"ID":"208dc340-b6f6-48d5-90aa-ccaedebfd4de","Type":"ContainerStarted","Data":"5563dc126221ea5d7fa2468dda483e6f889bfaa6294b6b64bdd28cbf1fc9dfb1"}
Jul 18 14:02:17.227111 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:17.227070    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:17.227307 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:17.227263    2577 patch_prober.go:28] interesting pod/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 18 14:02:17.227381 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:17.227315    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 18 14:02:18.227535 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:18.227499    2577 patch_prober.go:28] interesting pod/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 18 14:02:18.227930 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:18.227567    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 18 14:02:19.228217 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:19.228178    2577 patch_prober.go:28] interesting pod/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 18 14:02:19.228579 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:19.228240    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 18 14:02:20.227674 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:20.227627    2577 patch_prober.go:28] interesting pod/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 18 14:02:20.227851 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:20.227711    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 18 14:02:21.231517 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:21.231480    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:21.231892 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:21.231818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:21.232418 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:21.232401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:26.320564 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.320483    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"]
Jul 18 14:02:26.323837 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.323820    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:26.326168 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.326149    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:02:26.330252 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.330229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"]
Jul 18 14:02:26.405932 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.405896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b2789f-32b7-4a6e-8055-372283ae3ed5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr\" (UID: \"39b2789f-32b7-4a6e-8055-372283ae3ed5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:26.507286 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.507247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b2789f-32b7-4a6e-8055-372283ae3ed5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr\" (UID: \"39b2789f-32b7-4a6e-8055-372283ae3ed5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:26.507837 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.507817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b2789f-32b7-4a6e-8055-372283ae3ed5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr\" (UID: \"39b2789f-32b7-4a6e-8055-372283ae3ed5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:26.635677 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.635616    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:26.760860 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.760833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"]
Jul 18 14:02:26.762862 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:02:26.762834    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b2789f_32b7_4a6e_8055_372283ae3ed5.slice/crio-b9ee0018004728ee1de34fb87942f64b94ee48228e7ef76464942942f22be26f WatchSource:0}: Error finding container b9ee0018004728ee1de34fb87942f64b94ee48228e7ef76464942942f22be26f: Status 404 returned error can't find the container with id b9ee0018004728ee1de34fb87942f64b94ee48228e7ef76464942942f22be26f
Jul 18 14:02:26.764800 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:26.764779    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 14:02:27.607329 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:27.607295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr" event={"ID":"39b2789f-32b7-4a6e-8055-372283ae3ed5","Type":"ContainerStarted","Data":"b9ee0018004728ee1de34fb87942f64b94ee48228e7ef76464942942f22be26f"}
Jul 18 14:02:29.617931 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:29.617902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr" event={"ID":"39b2789f-32b7-4a6e-8055-372283ae3ed5","Type":"ContainerStarted","Data":"f0a693560e283ff1811a72e553de630c12a0f55ece13eccbfed00e3613914f7a"}
Jul 18 14:02:29.632756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:29.632712    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr" podStartSLOduration=1.175457098 podStartE2EDuration="3.632698045s" totalImagesPullingTime="2.457240947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:02:29.631109501 +0000 UTC m=+1799.287799629" watchObservedRunningTime="2026-07-18 14:02:29.632698045 +0000 UTC m=+1799.289388174"
Jul 18 14:02:30.623915 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:30.623885    2577 generic.go:356] "Generic (PLEG): container finished" podID="39b2789f-32b7-4a6e-8055-372283ae3ed5" containerID="f0a693560e283ff1811a72e553de630c12a0f55ece13eccbfed00e3613914f7a" exitCode=0
Jul 18 14:02:30.624291 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:30.623974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr" event={"ID":"39b2789f-32b7-4a6e-8055-372283ae3ed5","Type":"ContainerDied","Data":"f0a693560e283ff1811a72e553de630c12a0f55ece13eccbfed00e3613914f7a"}
Jul 18 14:02:30.837459 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:30.837428    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:02:30.843375 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:30.843351    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:02:30.843503 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:30.843465    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:02:30.849313 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:30.849296    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:02:31.757832 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:31.757810    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:31.854821 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:31.854790    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b2789f-32b7-4a6e-8055-372283ae3ed5-maas-api-service-ca\") pod \"39b2789f-32b7-4a6e-8055-372283ae3ed5\" (UID: \"39b2789f-32b7-4a6e-8055-372283ae3ed5\") "
Jul 18 14:02:31.855137 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:31.855115    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b2789f-32b7-4a6e-8055-372283ae3ed5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39b2789f-32b7-4a6e-8055-372283ae3ed5" (UID: "39b2789f-32b7-4a6e-8055-372283ae3ed5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:02:31.955394 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:31.955323    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b2789f-32b7-4a6e-8055-372283ae3ed5-maas-api-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:32.633892 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:32.633864    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"
Jul 18 14:02:32.634080 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:32.633861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr" event={"ID":"39b2789f-32b7-4a6e-8055-372283ae3ed5","Type":"ContainerDied","Data":"b9ee0018004728ee1de34fb87942f64b94ee48228e7ef76464942942f22be26f"}
Jul 18 14:02:32.634080 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:32.633992    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9ee0018004728ee1de34fb87942f64b94ee48228e7ef76464942942f22be26f"
Jul 18 14:02:36.693277 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:36.693239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:41.384894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.384864    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"]
Jul 18 14:02:41.387228 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.387204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:41.387487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.387468    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7d23fa97-snfkr"]
Jul 18 14:02:41.521543 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.521511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"]
Jul 18 14:02:41.521737 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.521711    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy" containerID="cri-o://5563dc126221ea5d7fa2468dda483e6f889bfaa6294b6b64bdd28cbf1fc9dfb1" gracePeriod=30
Jul 18 14:02:41.547053 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.547003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:41.791775 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:41.791740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:42.038152 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:42.038079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:42.133688 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:42.133594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:42.234937 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:42.234907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:42.577830 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:42.577788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:42.681351 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:42.681307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:42.802111 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:42.802078    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b2789f-32b7-4a6e-8055-372283ae3ed5" path="/var/lib/kubelet/pods/39b2789f-32b7-4a6e-8055-372283ae3ed5/volumes"
Jul 18 14:02:45.516712 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:45.516673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:45.618321 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:45.618290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:45.881777 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:45.881748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:46.202879 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.202790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:46.301640 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.301609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:02:46.704413 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.704379    2577 generic.go:356] "Generic (PLEG): container finished" podID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerID="5563dc126221ea5d7fa2468dda483e6f889bfaa6294b6b64bdd28cbf1fc9dfb1" exitCode=0
Jul 18 14:02:46.704877 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.704432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" event={"ID":"208dc340-b6f6-48d5-90aa-ccaedebfd4de","Type":"ContainerDied","Data":"5563dc126221ea5d7fa2468dda483e6f889bfaa6294b6b64bdd28cbf1fc9dfb1"}
Jul 18 14:02:46.704877 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.704490    2577 scope.go:117] "RemoveContainer" containerID="6e2580acba62a45f72f995165f1372cf9897bd0742ecf4c73a054337a594c786"
Jul 18 14:02:46.775123 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.775102    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:46.883973 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.883907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln9lr\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-kube-api-access-ln9lr\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.883973 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.883935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-socket\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.883973 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.883960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-envoy\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.883983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istiod-ca-cert\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884027    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-certs\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884067    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-credential-socket\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-data\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884151    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-token\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-podinfo\") pod \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\" (UID: \"208dc340-b6f6-48d5-90aa-ccaedebfd4de\") "
Jul 18 14:02:46.884248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:02:46.884590 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884456    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:02:46.884590 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884461    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.884590 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884488    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:02:46.884739 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884634    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:02:46.884947 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.884925    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-data" (OuterVolumeSpecName: "istio-data") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:02:46.886180 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.886161    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-kube-api-access-ln9lr" (OuterVolumeSpecName: "kube-api-access-ln9lr") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "kube-api-access-ln9lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:02:46.886277 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.886257    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:02:46.886568 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.886548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 14:02:46.886631 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.886611    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-token" (OuterVolumeSpecName: "istio-token") pod "208dc340-b6f6-48d5-90aa-ccaedebfd4de" (UID: "208dc340-b6f6-48d5-90aa-ccaedebfd4de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:02:46.985133 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985099    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-workload-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985133 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985126    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-credential-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985316 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985140    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-data\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985316 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985153    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985316 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985163    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-podinfo\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985316 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985174    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln9lr\" (UniqueName: \"kubernetes.io/projected/208dc340-b6f6-48d5-90aa-ccaedebfd4de-kube-api-access-ln9lr\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985316 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985185    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istio-envoy\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:46.985316 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:46.985196    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/208dc340-b6f6-48d5-90aa-ccaedebfd4de-istiod-ca-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:02:47.710270 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:47.710233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26" event={"ID":"208dc340-b6f6-48d5-90aa-ccaedebfd4de","Type":"ContainerDied","Data":"0dab1206025daca4ebf9bb59c821a0b20f1a1cd6b13fe0dbbbf5afe31553be3e"}
Jul 18 14:02:47.710270 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:47.710268    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"
Jul 18 14:02:47.710697 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:47.710320    2577 scope.go:117] "RemoveContainer" containerID="5563dc126221ea5d7fa2468dda483e6f889bfaa6294b6b64bdd28cbf1fc9dfb1"
Jul 18 14:02:47.730807 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:47.730785    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"]
Jul 18 14:02:47.736794 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:47.736771    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7d23fa97-openshift-default-6d9564f4cc-brg26"]
Jul 18 14:02:48.801322 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:02:48.801287    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" path="/var/lib/kubelet/pods/208dc340-b6f6-48d5-90aa-ccaedebfd4de/volumes"
Jul 18 14:03:10.499509 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499473    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"]
Jul 18 14:03:10.499882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499837    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy"
Jul 18 14:03:10.499882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499850    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy"
Jul 18 14:03:10.499882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499866    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b2789f-32b7-4a6e-8055-372283ae3ed5" containerName="revoke-keys"
Jul 18 14:03:10.499882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499872    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b2789f-32b7-4a6e-8055-372283ae3ed5" containerName="revoke-keys"
Jul 18 14:03:10.500011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499885    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy"
Jul 18 14:03:10.500011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499891    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy"
Jul 18 14:03:10.500011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499957    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy"
Jul 18 14:03:10.500011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499965    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="208dc340-b6f6-48d5-90aa-ccaedebfd4de" containerName="istio-proxy"
Jul 18 14:03:10.500011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.499974    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b2789f-32b7-4a6e-8055-372283ae3ed5" containerName="revoke-keys"
Jul 18 14:03:10.502828 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.502811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:10.505396 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.505374    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:03:10.509740 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.509715    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"]
Jul 18 14:03:10.571117 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.571081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a1634c-26a1-4c09-b248-c5020a10f07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6\" (UID: \"28a1634c-26a1-4c09-b248-c5020a10f07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:10.671814 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.671783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a1634c-26a1-4c09-b248-c5020a10f07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6\" (UID: \"28a1634c-26a1-4c09-b248-c5020a10f07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:10.672424 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.672397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a1634c-26a1-4c09-b248-c5020a10f07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6\" (UID: \"28a1634c-26a1-4c09-b248-c5020a10f07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:10.816287 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.816263    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:10.947240 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:10.947212    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"]
Jul 18 14:03:10.949540 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:03:10.949503    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a1634c_26a1_4c09_b248_c5020a10f07c.slice/crio-899697de67cc6fc93a4362e915861f3240dc125a045a652d725cdf01fec34dfc WatchSource:0}: Error finding container 899697de67cc6fc93a4362e915861f3240dc125a045a652d725cdf01fec34dfc: Status 404 returned error can't find the container with id 899697de67cc6fc93a4362e915861f3240dc125a045a652d725cdf01fec34dfc
Jul 18 14:03:11.819687 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:11.819656    2577 generic.go:356] "Generic (PLEG): container finished" podID="28a1634c-26a1-4c09-b248-c5020a10f07c" containerID="3938f61e7ac00672fe718c8cb69112186d1fe1e43dff2c2039426bd6ab4a7861" exitCode=0
Jul 18 14:03:11.820023 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:11.819711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6" event={"ID":"28a1634c-26a1-4c09-b248-c5020a10f07c","Type":"ContainerDied","Data":"3938f61e7ac00672fe718c8cb69112186d1fe1e43dff2c2039426bd6ab4a7861"}
Jul 18 14:03:11.820023 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:11.819760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6" event={"ID":"28a1634c-26a1-4c09-b248-c5020a10f07c","Type":"ContainerStarted","Data":"899697de67cc6fc93a4362e915861f3240dc125a045a652d725cdf01fec34dfc"}
Jul 18 14:03:12.953813 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:12.953785    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:13.093763 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:13.093687    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a1634c-26a1-4c09-b248-c5020a10f07c-maas-api-service-ca\") pod \"28a1634c-26a1-4c09-b248-c5020a10f07c\" (UID: \"28a1634c-26a1-4c09-b248-c5020a10f07c\") "
Jul 18 14:03:13.094027 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:13.094004    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a1634c-26a1-4c09-b248-c5020a10f07c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28a1634c-26a1-4c09-b248-c5020a10f07c" (UID: "28a1634c-26a1-4c09-b248-c5020a10f07c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:03:13.194372 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:13.194343    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a1634c-26a1-4c09-b248-c5020a10f07c-maas-api-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:03:13.830310 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:13.830281    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"
Jul 18 14:03:13.830479 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:13.830309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6" event={"ID":"28a1634c-26a1-4c09-b248-c5020a10f07c","Type":"ContainerDied","Data":"899697de67cc6fc93a4362e915861f3240dc125a045a652d725cdf01fec34dfc"}
Jul 18 14:03:13.830479 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:13.830353    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="899697de67cc6fc93a4362e915861f3240dc125a045a652d725cdf01fec34dfc"
Jul 18 14:03:20.870706 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:20.870667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:25.572232 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:25.572198    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"]
Jul 18 14:03:25.574680 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:25.574646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:25.574873 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:25.574854    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ecedf3e3-dn2j6"]
Jul 18 14:03:25.726335 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:25.726295    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:25.955440 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:25.955357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:26.185487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.185450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:26.446560 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.446516    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"]
Jul 18 14:03:26.446867 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.446853    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a1634c-26a1-4c09-b248-c5020a10f07c" containerName="revoke-keys"
Jul 18 14:03:26.446931 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.446869    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a1634c-26a1-4c09-b248-c5020a10f07c" containerName="revoke-keys"
Jul 18 14:03:26.446931 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.446924    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a1634c-26a1-4c09-b248-c5020a10f07c" containerName="revoke-keys"
Jul 18 14:03:26.453467 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.453442    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.457944 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.457914    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-56e18679-openshift-default-dockercfg-g7zdn\""
Jul 18 14:03:26.464545 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.464522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"]
Jul 18 14:03:26.473118 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.473096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:26.611327 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-token\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-podinfo\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-envoy\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-credential-socket\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-data\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-socket\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-certs\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8f8\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-kube-api-access-6q8f8\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.611750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.611717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f83c2f-24f4-4afa-83c0-d0147588bfba-istiod-ca-cert\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713209 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f83c2f-24f4-4afa-83c0-d0147588bfba-istiod-ca-cert\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713209 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-token\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713209 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-podinfo\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-envoy\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-credential-socket\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-data\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-socket\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-certs\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8f8\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-kube-api-access-6q8f8\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713766 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-credential-socket\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713806 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-data\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713921 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-socket\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.713980 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.713962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f83c2f-24f4-4afa-83c0-d0147588bfba-istiod-ca-cert\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.714189 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.714167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-certs\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.715690 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.715671    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-envoy\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.715690 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.715683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-podinfo\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.720708 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.720688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-token\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.720976 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.720957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8f8\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-kube-api-access-6q8f8\") pod \"e2e-ait-56e18679-openshift-default-75db894997-rkptl\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") " pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.730903 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.730877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:26.764948 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.764924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:26.803104 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.803066    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a1634c-26a1-4c09-b248-c5020a10f07c" path="/var/lib/kubelet/pods/28a1634c-26a1-4c09-b248-c5020a10f07c/volumes"
Jul 18 14:03:26.906076 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.906049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"]
Jul 18 14:03:26.907562 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:03:26.907537    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f83c2f_24f4_4afa_83c0_d0147588bfba.slice/crio-e3a54b4769de0fac07231b37b729b68b50b1c9b96a67c82966ad92a483c36d42 WatchSource:0}: Error finding container e3a54b4769de0fac07231b37b729b68b50b1c9b96a67c82966ad92a483c36d42: Status 404 returned error can't find the container with id e3a54b4769de0fac07231b37b729b68b50b1c9b96a67c82966ad92a483c36d42
Jul 18 14:03:26.909512 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.909479    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:03:26.909623 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.909553    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:03:26.909623 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:26.909594    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:03:27.896352 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:27.896315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" event={"ID":"62f83c2f-24f4-4afa-83c0-d0147588bfba","Type":"ContainerStarted","Data":"2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"}
Jul 18 14:03:27.896826 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:27.896370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" event={"ID":"62f83c2f-24f4-4afa-83c0-d0147588bfba","Type":"ContainerStarted","Data":"e3a54b4769de0fac07231b37b729b68b50b1c9b96a67c82966ad92a483c36d42"}
Jul 18 14:03:27.915729 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:27.915649    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podStartSLOduration=1.915631512 podStartE2EDuration="1.915631512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:03:27.915254767 +0000 UTC m=+1857.571944896" watchObservedRunningTime="2026-07-18 14:03:27.915631512 +0000 UTC m=+1857.572321642"
Jul 18 14:03:28.765866 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:28.765829    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:28.770638 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:28.770615    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:28.901695 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:28.901660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:28.902587 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:28.902569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:29.228499 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:29.228439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:29.328291 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:29.328258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:29.447572 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:29.447542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:29.558497 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:29.558459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:29.875504 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:29.875421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:29.973602 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:29.973558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:03:44.061682 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.061596    2577 patch_prober.go:28] interesting pod/e2e-ait-56e18679-openshift-default-75db894997-rkptl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:47404->10.134.0.40:15021: read: connection reset by peer" start-of-body=
Jul 18 14:03:44.062067 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.061680    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:47404->10.134.0.40:15021: read: connection reset by peer"
Jul 18 14:03:44.980493 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.980455    2577 generic.go:356] "Generic (PLEG): container finished" podID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerID="2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee" exitCode=0
Jul 18 14:03:44.980669 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.980528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" event={"ID":"62f83c2f-24f4-4afa-83c0-d0147588bfba","Type":"ContainerDied","Data":"2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"}
Jul 18 14:03:44.980937 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.980923    2577 scope.go:117] "RemoveContainer" containerID="2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"
Jul 18 14:03:44.981954 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.981922    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:03:44.982071 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.982010    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:03:44.982143 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:44.982086    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:03:45.988428 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:45.988386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" event={"ID":"62f83c2f-24f4-4afa-83c0-d0147588bfba","Type":"ContainerStarted","Data":"10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c"}
Jul 18 14:03:46.765701 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:46.765661    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:46.765922 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:46.765897    2577 patch_prober.go:28] interesting pod/e2e-ait-56e18679-openshift-default-75db894997-rkptl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 18 14:03:46.765974 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:46.765948    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 18 14:03:47.765945 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:47.765907    2577 patch_prober.go:28] interesting pod/e2e-ait-56e18679-openshift-default-75db894997-rkptl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 18 14:03:47.766370 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:47.765964    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 18 14:03:48.766329 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:48.766289    2577 patch_prober.go:28] interesting pod/e2e-ait-56e18679-openshift-default-75db894997-rkptl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 18 14:03:48.766732 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:48.766353    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 18 14:03:49.765459 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:49.765420    2577 patch_prober.go:28] interesting pod/e2e-ait-56e18679-openshift-default-75db894997-rkptl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 18 14:03:49.765646 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:49.765480    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 18 14:03:50.768980 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:50.768948    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:50.769404 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:50.769259    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:50.770095 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:50.770076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:03:54.946294 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:54.946208    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"]
Jul 18 14:03:54.950134 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:54.950115    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:54.952356 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:54.952328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:03:54.955549 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:54.955525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"]
Jul 18 14:03:55.077132 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:55.077087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-56e18679-vlgqj\" (UID: \"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:55.177978 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:55.177938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-56e18679-vlgqj\" (UID: \"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:55.178599 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:55.178579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-56e18679-vlgqj\" (UID: \"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:55.269018 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:55.268930    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:55.411453 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:55.411425    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"]
Jul 18 14:03:55.414446 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:03:55.414413    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7521a86_88f3_43ae_9e8b_fb54f3b9bce7.slice/crio-a6330b27d27e5fb28f2bc005014534650a012b2b30a764cbb942c2825a04af73 WatchSource:0}: Error finding container a6330b27d27e5fb28f2bc005014534650a012b2b30a764cbb942c2825a04af73: Status 404 returned error can't find the container with id a6330b27d27e5fb28f2bc005014534650a012b2b30a764cbb942c2825a04af73
Jul 18 14:03:56.049169 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:56.049134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj" event={"ID":"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7","Type":"ContainerStarted","Data":"f69e9f81e63f07331ab431afe03c29beca2580f55c51842a5c8478b90fb4692f"}
Jul 18 14:03:56.049547 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:56.049189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj" event={"ID":"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7","Type":"ContainerStarted","Data":"a6330b27d27e5fb28f2bc005014534650a012b2b30a764cbb942c2825a04af73"}
Jul 18 14:03:56.064092 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:56.064015    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj" podStartSLOduration=2.064001042 podStartE2EDuration="2.064001042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:03:56.063061262 +0000 UTC m=+1885.719751391" watchObservedRunningTime="2026-07-18 14:03:56.064001042 +0000 UTC m=+1885.720691170"
Jul 18 14:03:57.055087 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:57.055024    2577 generic.go:356] "Generic (PLEG): container finished" podID="b7521a86-88f3-43ae-9e8b-fb54f3b9bce7" containerID="f69e9f81e63f07331ab431afe03c29beca2580f55c51842a5c8478b90fb4692f" exitCode=0
Jul 18 14:03:57.055087 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:57.055068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj" event={"ID":"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7","Type":"ContainerDied","Data":"f69e9f81e63f07331ab431afe03c29beca2580f55c51842a5c8478b90fb4692f"}
Jul 18 14:03:58.193940 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:58.193915    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:58.308136 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:58.308099    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7-maas-api-service-ca\") pod \"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7\" (UID: \"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7\") "
Jul 18 14:03:58.308451 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:58.308426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7521a86-88f3-43ae-9e8b-fb54f3b9bce7" (UID: "b7521a86-88f3-43ae-9e8b-fb54f3b9bce7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:03:58.409647 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:58.409567    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7-maas-api-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:03:59.066325 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:59.066244    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"
Jul 18 14:03:59.066325 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:59.066279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj" event={"ID":"b7521a86-88f3-43ae-9e8b-fb54f3b9bce7","Type":"ContainerDied","Data":"a6330b27d27e5fb28f2bc005014534650a012b2b30a764cbb942c2825a04af73"}
Jul 18 14:03:59.066514 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:03:59.066330    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6330b27d27e5fb28f2bc005014534650a012b2b30a764cbb942c2825a04af73"
Jul 18 14:04:05.327081 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:05.327025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:10.016411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:10.016365    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"]
Jul 18 14:04:10.021491 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:10.021449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:10.021749 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:10.021685    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-56e18679-vlgqj"]
Jul 18 14:04:10.801560 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:10.801525    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7521a86-88f3-43ae-9e8b-fb54f3b9bce7" path="/var/lib/kubelet/pods/b7521a86-88f3-43ae-9e8b-fb54f3b9bce7/volumes"
Jul 18 14:04:11.142642 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.142563    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"]
Jul 18 14:04:11.143110 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.142785    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy" containerID="cri-o://10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c" gracePeriod=30
Jul 18 14:04:11.164444 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.164401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:11.375295 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.375213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:11.596651 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.596618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:11.880341 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.880262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:11.992210 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:11.992179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:13.917850 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:13.917805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:13.998389 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:13.998349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:14.061936 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:14.061906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:14.408252 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:14.408216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:14.541208 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:14.541173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:14.639370 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:14.639336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:16.411471 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.411397    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:04:16.473248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473203    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-token\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-certs\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473296    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-podinfo\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473315    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-data\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473337    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f83c2f-24f4-4afa-83c0-d0147588bfba-istiod-ca-cert\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473360    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-envoy\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473379    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-credential-socket\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-socket\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.473809 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473708    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:04:16.473809 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473761    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:04:16.473809 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473794    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f83c2f-24f4-4afa-83c0-d0147588bfba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:04:16.473963 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473805    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.473963 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.473832    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:04:16.474514 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.474474    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-data" (OuterVolumeSpecName: "istio-data") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:04:16.475782 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.475754    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-token" (OuterVolumeSpecName: "istio-token") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:04:16.475782 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.475761    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 14:04:16.475782 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.475777    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:04:16.574200 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q8f8\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-kube-api-access-6q8f8\") pod \"62f83c2f-24f4-4afa-83c0-d0147588bfba\" (UID: \"62f83c2f-24f4-4afa-83c0-d0147588bfba\") "
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574334    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574351    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-podinfo\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574363    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-data\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574375    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f83c2f-24f4-4afa-83c0-d0147588bfba-istiod-ca-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574386    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-istio-envoy\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574398    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-credential-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.574411 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.574411    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f83c2f-24f4-4afa-83c0-d0147588bfba-workload-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:16.576299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.576272    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-kube-api-access-6q8f8" (OuterVolumeSpecName: "kube-api-access-6q8f8") pod "62f83c2f-24f4-4afa-83c0-d0147588bfba" (UID: "62f83c2f-24f4-4afa-83c0-d0147588bfba"). InnerVolumeSpecName "kube-api-access-6q8f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:04:16.674927 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:16.674831    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q8f8\" (UniqueName: \"kubernetes.io/projected/62f83c2f-24f4-4afa-83c0-d0147588bfba-kube-api-access-6q8f8\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:17.157577 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.157539    2577 generic.go:356] "Generic (PLEG): container finished" podID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerID="10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c" exitCode=0
Jul 18 14:04:17.157748 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.157593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" event={"ID":"62f83c2f-24f4-4afa-83c0-d0147588bfba","Type":"ContainerDied","Data":"10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c"}
Jul 18 14:04:17.157748 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.157604    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"
Jul 18 14:04:17.157748 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.157642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl" event={"ID":"62f83c2f-24f4-4afa-83c0-d0147588bfba","Type":"ContainerDied","Data":"e3a54b4769de0fac07231b37b729b68b50b1c9b96a67c82966ad92a483c36d42"}
Jul 18 14:04:17.157748 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.157667    2577 scope.go:117] "RemoveContainer" containerID="10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c"
Jul 18 14:04:17.166588 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.166566    2577 scope.go:117] "RemoveContainer" containerID="2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"
Jul 18 14:04:17.175267 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.175242    2577 scope.go:117] "RemoveContainer" containerID="10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c"
Jul 18 14:04:17.175570 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:04:17.175543    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c\": container with ID starting with 10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c not found: ID does not exist" containerID="10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c"
Jul 18 14:04:17.175637 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.175581    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c"} err="failed to get container status \"10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c\": rpc error: code = NotFound desc = could not find container \"10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c\": container with ID starting with 10fab92e1df06130b3bcfcd5f00ddedc50a812e322df695ee51f0e14f79a232c not found: ID does not exist"
Jul 18 14:04:17.175637 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.175606    2577 scope.go:117] "RemoveContainer" containerID="2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"
Jul 18 14:04:17.175756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.175740    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"]
Jul 18 14:04:17.175849 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:04:17.175829    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee\": container with ID starting with 2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee not found: ID does not exist" containerID="2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"
Jul 18 14:04:17.175890 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.175856    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee"} err="failed to get container status \"2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee\": rpc error: code = NotFound desc = could not find container \"2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee\": container with ID starting with 2531f3671705b08c3e107d1ec354d69f7a1b983371f19a8a1c2190ce51956cee not found: ID does not exist"
Jul 18 14:04:17.180780 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:17.180756    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-56e18679-openshift-default-75db894997-rkptl"]
Jul 18 14:04:18.802100 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:18.802063    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" path="/var/lib/kubelet/pods/62f83c2f-24f4-4afa-83c0-d0147588bfba/volumes"
Jul 18 14:04:38.778325 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778289    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"]
Jul 18 14:04:38.778699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778615    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7521a86-88f3-43ae-9e8b-fb54f3b9bce7" containerName="revoke-keys"
Jul 18 14:04:38.778699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778626    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7521a86-88f3-43ae-9e8b-fb54f3b9bce7" containerName="revoke-keys"
Jul 18 14:04:38.778699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778642    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy"
Jul 18 14:04:38.778699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778647    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy"
Jul 18 14:04:38.778699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778654    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy"
Jul 18 14:04:38.778699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778661    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy"
Jul 18 14:04:38.778887 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778717    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy"
Jul 18 14:04:38.778887 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.778725    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7521a86-88f3-43ae-9e8b-fb54f3b9bce7" containerName="revoke-keys"
Jul 18 14:04:38.781596 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.781580    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:38.784217 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.784196    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:04:38.789731 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.789705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"]
Jul 18 14:04:38.841976 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.841943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0589263e-6b9d-4f3c-bc52-fab32345c62e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ea60214d-8pg24\" (UID: \"0589263e-6b9d-4f3c-bc52-fab32345c62e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:38.943020 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.942986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0589263e-6b9d-4f3c-bc52-fab32345c62e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ea60214d-8pg24\" (UID: \"0589263e-6b9d-4f3c-bc52-fab32345c62e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:38.943563 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:38.943545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0589263e-6b9d-4f3c-bc52-fab32345c62e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ea60214d-8pg24\" (UID: \"0589263e-6b9d-4f3c-bc52-fab32345c62e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:39.012242 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:39.012211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:39.093598 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:39.093534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:39.222494 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:39.222464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"]
Jul 18 14:04:39.224402 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:04:39.224375    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0589263e_6b9d_4f3c_bc52_fab32345c62e.slice/crio-0369ce1fb119e38334c6d2145f3583f3a7d438c1583dc475f335b4856d2b2895 WatchSource:0}: Error finding container 0369ce1fb119e38334c6d2145f3583f3a7d438c1583dc475f335b4856d2b2895: Status 404 returned error can't find the container with id 0369ce1fb119e38334c6d2145f3583f3a7d438c1583dc475f335b4856d2b2895
Jul 18 14:04:39.264848 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:39.264820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24" event={"ID":"0589263e-6b9d-4f3c-bc52-fab32345c62e","Type":"ContainerStarted","Data":"0369ce1fb119e38334c6d2145f3583f3a7d438c1583dc475f335b4856d2b2895"}
Jul 18 14:04:40.271126 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:40.271008    2577 generic.go:356] "Generic (PLEG): container finished" podID="0589263e-6b9d-4f3c-bc52-fab32345c62e" containerID="d5309bfd7177bece7a177ecf011fcf72c8bc341f963f456bcac5cd433be5ec63" exitCode=0
Jul 18 14:04:40.271126 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:40.271096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24" event={"ID":"0589263e-6b9d-4f3c-bc52-fab32345c62e","Type":"ContainerDied","Data":"d5309bfd7177bece7a177ecf011fcf72c8bc341f963f456bcac5cd433be5ec63"}
Jul 18 14:04:41.413679 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:41.413652    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:41.464501 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:41.464476    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0589263e-6b9d-4f3c-bc52-fab32345c62e-maas-api-service-ca\") pod \"0589263e-6b9d-4f3c-bc52-fab32345c62e\" (UID: \"0589263e-6b9d-4f3c-bc52-fab32345c62e\") "
Jul 18 14:04:41.464799 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:41.464778    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0589263e-6b9d-4f3c-bc52-fab32345c62e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0589263e-6b9d-4f3c-bc52-fab32345c62e" (UID: "0589263e-6b9d-4f3c-bc52-fab32345c62e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:04:41.565279 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:41.565213    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0589263e-6b9d-4f3c-bc52-fab32345c62e-maas-api-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:04:42.289821 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:42.289787    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"
Jul 18 14:04:42.290015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:42.289789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24" event={"ID":"0589263e-6b9d-4f3c-bc52-fab32345c62e","Type":"ContainerDied","Data":"0369ce1fb119e38334c6d2145f3583f3a7d438c1583dc475f335b4856d2b2895"}
Jul 18 14:04:42.290015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:42.289972    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0369ce1fb119e38334c6d2145f3583f3a7d438c1583dc475f335b4856d2b2895"
Jul 18 14:04:49.165901 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:49.165863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:52.053976 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:52.053941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:52.156988 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:52.156958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:53.845773 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:53.845738    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"]
Jul 18 14:04:53.848436 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:53.848408    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ea60214d-8pg24"]
Jul 18 14:04:53.850015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:53.849981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:53.994463 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:53.994432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:54.226015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:54.225929    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:54.691610 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:54.691561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:54.792021 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:54.791987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:54.801565 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:54.801538    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0589263e-6b9d-4f3c-bc52-fab32345c62e" path="/var/lib/kubelet/pods/0589263e-6b9d-4f3c-bc52-fab32345c62e/volumes"
Jul 18 14:04:54.893737 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:54.893708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:04:56.258623 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:04:56.258585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:02.027494 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:02.027458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:02.084598 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:02.084567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:02.155179 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:02.155147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:02.255846 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:02.255821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:02.355894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:02.355828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:02.722878 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:02.722808    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:05.976661 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:05.976622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:06.077209 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:06.077177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:28.625692 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:28.625657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:34.591776 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:34.591735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:34.695236 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:34.695205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:34.791578 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:34.791549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:50.623474 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:50.623435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:56.517423 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:56.517390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:56.573650 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:56.573620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:56.662186 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:56.662160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:05:56.760271 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:05:56.760244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:07.221977 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:07.221943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:08.000447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:08.000408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:14.469752 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:14.469713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:14.577755 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:14.577727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:14.670049 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:14.669995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:14.769933 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:14.769847    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:15.511224 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:15.511187    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:15.609882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:15.609852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:15.711409 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:15.711379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:17.217634 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:17.217602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:22.876744 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:22.876684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:22.978004 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:22.977969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:23.522136 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:23.522097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:29.306933 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:29.306900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:29.368792 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:29.368763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:29.540149 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:29.540119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:29.842394 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:29.842358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:30.262511 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:30.262419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:32.426662 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.426628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:32.526782 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.526750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:32.532993 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.532963    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"]
Jul 18 14:06:32.533482 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.533464    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0589263e-6b9d-4f3c-bc52-fab32345c62e" containerName="revoke-keys"
Jul 18 14:06:32.533567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.533484    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0589263e-6b9d-4f3c-bc52-fab32345c62e" containerName="revoke-keys"
Jul 18 14:06:32.533625 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.533577    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0589263e-6b9d-4f3c-bc52-fab32345c62e" containerName="revoke-keys"
Jul 18 14:06:32.533625 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.533591    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f83c2f-24f4-4afa-83c0-d0147588bfba" containerName="istio-proxy"
Jul 18 14:06:32.536953 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.536928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.539174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.539152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2d50f1fd-serving-cert\""
Jul 18 14:06:32.539174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.539165    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-97gtb\""
Jul 18 14:06:32.545587 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.545565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"]
Jul 18 14:06:32.556895 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.556871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h2zx\" (UniqueName: \"kubernetes.io/projected/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-kube-api-access-2h2zx\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.557006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.556910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-openshift-service-ca\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.557006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.556937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-maas-api-tls\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.627473 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.627447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:32.658095 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.658070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h2zx\" (UniqueName: \"kubernetes.io/projected/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-kube-api-access-2h2zx\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.658227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.658113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-openshift-service-ca\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.658227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.658134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-maas-api-tls\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.658812 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.658789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-openshift-service-ca\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.660576 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.660558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-maas-api-tls\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.665821 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.665801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h2zx\" (UniqueName: \"kubernetes.io/projected/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-kube-api-access-2h2zx\") pod \"maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.728328 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.728248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:32.848998 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.848967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:32.983217 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:32.983174    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"]
Jul 18 14:06:33.087801 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:33.087761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:33.188015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:33.187979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:33.833871 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:33.833834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x" event={"ID":"b5f7404e-5bd0-476d-9f4a-9578a55e75fc","Type":"ContainerStarted","Data":"01e928ff165a32eb6324a07c3c990c027036076ff848ce7db21790b436a2caba"}
Jul 18 14:06:34.840363 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:34.840322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x" event={"ID":"b5f7404e-5bd0-476d-9f4a-9578a55e75fc","Type":"ContainerStarted","Data":"44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1"}
Jul 18 14:06:34.840735 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:34.840464    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:34.857807 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:34.857760    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x" podStartSLOduration=1.5224367060000001 podStartE2EDuration="2.857747196s" totalImagesPullingTime="1.33531049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:06:34.856433122 +0000 UTC m=+2044.513123252" watchObservedRunningTime="2026-07-18 14:06:34.857747196 +0000 UTC m=+2044.514437324"
Jul 18 14:06:36.896868 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:36.896821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:37.517072 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:37.517024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:37.607742 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:37.607712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:37.916455 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:37.916424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:40.850897 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:40.850859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:43.804174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:43.804139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:43.868926 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:43.868896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:44.144722 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.144637    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"]
Jul 18 14:06:44.148558 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.148528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:44.150731 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.150712    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gkls\""
Jul 18 14:06:44.156231 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.156210    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"]
Jul 18 14:06:44.255368 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.255336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc208a5-0689-4a62-aa84-d8515b844bea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r\" (UID: \"4cc208a5-0689-4a62-aa84-d8515b844bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:44.356219 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.356190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc208a5-0689-4a62-aa84-d8515b844bea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r\" (UID: \"4cc208a5-0689-4a62-aa84-d8515b844bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:44.356819 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.356797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc208a5-0689-4a62-aa84-d8515b844bea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r\" (UID: \"4cc208a5-0689-4a62-aa84-d8515b844bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:44.460755 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.460678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:44.598661 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.598634    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"]
Jul 18 14:06:44.600925 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:06:44.600898    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc208a5_0689_4a62_aa84_d8515b844bea.slice/crio-17df0004cbfcb407d538ef60a87a7868169c23532dbc9ed8ac995b1716d46bf7 WatchSource:0}: Error finding container 17df0004cbfcb407d538ef60a87a7868169c23532dbc9ed8ac995b1716d46bf7: Status 404 returned error can't find the container with id 17df0004cbfcb407d538ef60a87a7868169c23532dbc9ed8ac995b1716d46bf7
Jul 18 14:06:44.887116 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:44.887067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r" event={"ID":"4cc208a5-0689-4a62-aa84-d8515b844bea","Type":"ContainerStarted","Data":"17df0004cbfcb407d538ef60a87a7868169c23532dbc9ed8ac995b1716d46bf7"}
Jul 18 14:06:45.892987 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:45.892953    2577 generic.go:356] "Generic (PLEG): container finished" podID="4cc208a5-0689-4a62-aa84-d8515b844bea" containerID="12ff6132828484961fe1299c99d4a3413a40b041f751c4880e4755d998064501" exitCode=0
Jul 18 14:06:45.893377 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:45.893017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r" event={"ID":"4cc208a5-0689-4a62-aa84-d8515b844bea","Type":"ContainerDied","Data":"12ff6132828484961fe1299c99d4a3413a40b041f751c4880e4755d998064501"}
Jul 18 14:06:47.027018 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.026993    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:47.082588 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.082558    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc208a5-0689-4a62-aa84-d8515b844bea-maas-api-service-ca\") pod \"4cc208a5-0689-4a62-aa84-d8515b844bea\" (UID: \"4cc208a5-0689-4a62-aa84-d8515b844bea\") "
Jul 18 14:06:47.082883 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.082859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cc208a5-0689-4a62-aa84-d8515b844bea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4cc208a5-0689-4a62-aa84-d8515b844bea" (UID: "4cc208a5-0689-4a62-aa84-d8515b844bea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:06:47.183727 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.183659    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc208a5-0689-4a62-aa84-d8515b844bea-maas-api-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:06:47.903210 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.903183    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"
Jul 18 14:06:47.903357 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.903230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r" event={"ID":"4cc208a5-0689-4a62-aa84-d8515b844bea","Type":"ContainerDied","Data":"17df0004cbfcb407d538ef60a87a7868169c23532dbc9ed8ac995b1716d46bf7"}
Jul 18 14:06:47.903357 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:47.903273    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17df0004cbfcb407d538ef60a87a7868169c23532dbc9ed8ac995b1716d46bf7"
Jul 18 14:06:54.490445 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.490369    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"]
Jul 18 14:06:54.490896 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.490605    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x" podUID="b5f7404e-5bd0-476d-9f4a-9578a55e75fc" containerName="maas-api" containerID="cri-o://44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1" gracePeriod=30
Jul 18 14:06:54.529275 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.529248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:54.739313 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.739288    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:54.852749 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.852717    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h2zx\" (UniqueName: \"kubernetes.io/projected/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-kube-api-access-2h2zx\") pod \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") "
Jul 18 14:06:54.852940 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.852783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-openshift-service-ca\") pod \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") "
Jul 18 14:06:54.852940 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.852810    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-maas-api-tls\") pod \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\" (UID: \"b5f7404e-5bd0-476d-9f4a-9578a55e75fc\") "
Jul 18 14:06:54.853212 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.853186    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5f7404e-5bd0-476d-9f4a-9578a55e75fc" (UID: "b5f7404e-5bd0-476d-9f4a-9578a55e75fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:06:54.854964 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.854933    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5f7404e-5bd0-476d-9f4a-9578a55e75fc" (UID: "b5f7404e-5bd0-476d-9f4a-9578a55e75fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:06:54.855085 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.854995    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-kube-api-access-2h2zx" (OuterVolumeSpecName: "kube-api-access-2h2zx") pod "b5f7404e-5bd0-476d-9f4a-9578a55e75fc" (UID: "b5f7404e-5bd0-476d-9f4a-9578a55e75fc"). InnerVolumeSpecName "kube-api-access-2h2zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:06:54.937703 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.937674    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5f7404e-5bd0-476d-9f4a-9578a55e75fc" containerID="44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1" exitCode=0
Jul 18 14:06:54.937810 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.937717    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"
Jul 18 14:06:54.937810 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.937731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x" event={"ID":"b5f7404e-5bd0-476d-9f4a-9578a55e75fc","Type":"ContainerDied","Data":"44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1"}
Jul 18 14:06:54.937810 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.937772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x" event={"ID":"b5f7404e-5bd0-476d-9f4a-9578a55e75fc","Type":"ContainerDied","Data":"01e928ff165a32eb6324a07c3c990c027036076ff848ce7db21790b436a2caba"}
Jul 18 14:06:54.937810 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.937794    2577 scope.go:117] "RemoveContainer" containerID="44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1"
Jul 18 14:06:54.946511 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.946493    2577 scope.go:117] "RemoveContainer" containerID="44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1"
Jul 18 14:06:54.946761 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:06:54.946742    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1\": container with ID starting with 44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1 not found: ID does not exist" containerID="44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1"
Jul 18 14:06:54.946817 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.946767    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1"} err="failed to get container status \"44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1\": rpc error: code = NotFound desc = could not find container \"44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1\": container with ID starting with 44ad1ab560c9ebd000ce87bb29ff3d5dbd16364bd8c4b386ba6b2d6bc573d1e1 not found: ID does not exist"
Jul 18 14:06:54.953782 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.953764    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:06:54.953855 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.953784    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:06:54.953855 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.953793    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h2zx\" (UniqueName: \"kubernetes.io/projected/b5f7404e-5bd0-476d-9f4a-9578a55e75fc-kube-api-access-2h2zx\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:06:54.957706 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.957680    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"]
Jul 18 14:06:54.961323 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:54.961303    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d50f1fd-58c5cf7dbc-w7g4x"]
Jul 18 14:06:56.802175 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:56.802143    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f7404e-5bd0-476d-9f4a-9578a55e75fc" path="/var/lib/kubelet/pods/b5f7404e-5bd0-476d-9f4a-9578a55e75fc/volumes"
Jul 18 14:06:59.211287 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:59.211251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:06:59.211736 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:59.211537    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"]
Jul 18 14:06:59.213113 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:59.213093    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2d50f1fd-v447r"]
Jul 18 14:06:59.560332 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:06:59.560297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:00.802140 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:00.802106    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc208a5-0689-4a62-aa84-d8515b844bea" path="/var/lib/kubelet/pods/4cc208a5-0689-4a62-aa84-d8515b844bea/volumes"
Jul 18 14:07:05.219912 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:05.219878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:05.278115 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:05.278076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:05.462295 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:05.462259    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:05.696341 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:05.696309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:05.803640 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:05.803612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:05.904867 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:05.904820    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:06.017011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:06.016933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:11.590819 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:11.590787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:11.696211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:11.696172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:11.792728 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:11.792693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:11.933160 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:11.933081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:17.458228 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:17.458195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:17.519778 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:17.519741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:24.185224 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.185194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:24.459892 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.459796    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"]
Jul 18 14:07:24.460458 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.460431    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f7404e-5bd0-476d-9f4a-9578a55e75fc" containerName="maas-api"
Jul 18 14:07:24.460458 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.460458    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f7404e-5bd0-476d-9f4a-9578a55e75fc" containerName="maas-api"
Jul 18 14:07:24.460651 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.460480    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc208a5-0689-4a62-aa84-d8515b844bea" containerName="revoke-keys"
Jul 18 14:07:24.460651 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.460490    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc208a5-0689-4a62-aa84-d8515b844bea" containerName="revoke-keys"
Jul 18 14:07:24.460651 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.460573    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc208a5-0689-4a62-aa84-d8515b844bea" containerName="revoke-keys"
Jul 18 14:07:24.460651 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.460593    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f7404e-5bd0-476d-9f4a-9578a55e75fc" containerName="maas-api"
Jul 18 14:07:24.463972 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.463954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.466130 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.466113    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ffbfeb3b-openshift-default-dockercfg-9bg8b\""
Jul 18 14:07:24.473260 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.473237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"]
Jul 18 14:07:24.491736 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-socket\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.491882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-token\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.491882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-certs\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.491882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-podinfo\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.491882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl862\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-kube-api-access-nl862\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.492133 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-credential-socket\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.492133 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-envoy\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.492133 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.491976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7456631b-3875-4769-8a54-4bf85fadbc6e-istiod-ca-cert\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.492133 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.492002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-data\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.496331 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.496301    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:24.592597 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-socket\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592597 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-token\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-certs\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-podinfo\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl862\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-kube-api-access-nl862\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-credential-socket\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-envoy\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7456631b-3875-4769-8a54-4bf85fadbc6e-istiod-ca-cert\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.592865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.592829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-data\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.593257 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.593063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-socket\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.593257 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.593120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-certs\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.593366 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.593282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-data\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.593428 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.593388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-credential-socket\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.593818 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.593790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7456631b-3875-4769-8a54-4bf85fadbc6e-istiod-ca-cert\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.595264 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.595238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-envoy\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.595445 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.595427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-podinfo\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.600945 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.600917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-token\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.601032 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.600950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl862\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-kube-api-access-nl862\") pod \"e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") " pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.776185 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.776146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:24.809132 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.809096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:24.914357 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.914331    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"]
Jul 18 14:07:24.916558 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:07:24.916531    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7456631b_3875_4769_8a54_4bf85fadbc6e.slice/crio-0e5af10bb1c264d0a5907ba7419dea7747b8acd4b815fbbd18cc70a32d6cac85 WatchSource:0}: Error finding container 0e5af10bb1c264d0a5907ba7419dea7747b8acd4b815fbbd18cc70a32d6cac85: Status 404 returned error can't find the container with id 0e5af10bb1c264d0a5907ba7419dea7747b8acd4b815fbbd18cc70a32d6cac85
Jul 18 14:07:24.919201 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.919161    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:07:24.919291 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.919258    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:07:24.919342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:24.919300    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:07:25.083573 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:25.083466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" event={"ID":"7456631b-3875-4769-8a54-4bf85fadbc6e","Type":"ContainerStarted","Data":"ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"}
Jul 18 14:07:25.083573 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:25.083527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" event={"ID":"7456631b-3875-4769-8a54-4bf85fadbc6e","Type":"ContainerStarted","Data":"0e5af10bb1c264d0a5907ba7419dea7747b8acd4b815fbbd18cc70a32d6cac85"}
Jul 18 14:07:25.103074 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:25.103013    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" podStartSLOduration=1.102997981 podStartE2EDuration="1.102997981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:07:25.101056647 +0000 UTC m=+2094.757746774" watchObservedRunningTime="2026-07-18 14:07:25.102997981 +0000 UTC m=+2094.759688110"
Jul 18 14:07:25.776699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:25.776665    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:25.781344 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:25.781319    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:26.088923 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:26.088844    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:26.089730 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:26.089712    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:26.965447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:26.965407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:27.037416 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.037377    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"]
Jul 18 14:07:27.039556 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.039526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:27.041328 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.041306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.043693 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.043674    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ffbfeb3b-serving-cert\""
Jul 18 14:07:27.043794 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.043690    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-97gtb\""
Jul 18 14:07:27.051330 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.051294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"]
Jul 18 14:07:27.103682 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.103650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:27.111901 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.111873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a1623d-1e97-4179-8ddf-7ab17fc3e837-openshift-service-ca\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.111901 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.111902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.112074 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.112027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82fk8\" (UniqueName: \"kubernetes.io/projected/20a1623d-1e97-4179-8ddf-7ab17fc3e837-kube-api-access-82fk8\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.212473 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.212437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82fk8\" (UniqueName: \"kubernetes.io/projected/20a1623d-1e97-4179-8ddf-7ab17fc3e837-kube-api-access-82fk8\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.212658 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.212489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a1623d-1e97-4179-8ddf-7ab17fc3e837-openshift-service-ca\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.212658 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.212507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.212658 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:07:27.212599    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-serving-cert: secret "maas-api-e2e-mt-ffbfeb3b-serving-cert" not found
Jul 18 14:07:27.212658 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:07:27.212652    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls podName:20a1623d-1e97-4179-8ddf-7ab17fc3e837 nodeName:}" failed. No retries permitted until 2026-07-18 14:07:27.712635034 +0000 UTC m=+2097.369325142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls") pod "maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" (UID: "20a1623d-1e97-4179-8ddf-7ab17fc3e837") : secret "maas-api-e2e-mt-ffbfeb3b-serving-cert" not found
Jul 18 14:07:27.213168 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.213142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a1623d-1e97-4179-8ddf-7ab17fc3e837-openshift-service-ca\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.221139 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.221081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82fk8\" (UniqueName: \"kubernetes.io/projected/20a1623d-1e97-4179-8ddf-7ab17fc3e837-kube-api-access-82fk8\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.502665 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.502588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:27.604830 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.604801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:27.715831 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.715790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.718371 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.718346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls\") pod \"maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:27.720299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.720276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:27.955059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:27.954998    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:28.091143 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:28.091119    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"]
Jul 18 14:07:28.092711 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:07:28.092686    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a1623d_1e97_4179_8ddf_7ab17fc3e837.slice/crio-400e134c0ea5591b2ae4db8a791701cdc22ff71c1367a440913a4dc19368717e WatchSource:0}: Error finding container 400e134c0ea5591b2ae4db8a791701cdc22ff71c1367a440913a4dc19368717e: Status 404 returned error can't find the container with id 400e134c0ea5591b2ae4db8a791701cdc22ff71c1367a440913a4dc19368717e
Jul 18 14:07:28.094108 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:28.094089    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 14:07:28.099547 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:28.099523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" event={"ID":"20a1623d-1e97-4179-8ddf-7ab17fc3e837","Type":"ContainerStarted","Data":"400e134c0ea5591b2ae4db8a791701cdc22ff71c1367a440913a4dc19368717e"}
Jul 18 14:07:29.105756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:29.105719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" event={"ID":"20a1623d-1e97-4179-8ddf-7ab17fc3e837","Type":"ContainerStarted","Data":"8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc"}
Jul 18 14:07:29.106157 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:29.105793    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:29.123835 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:29.123786    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" podStartSLOduration=1.755611419 podStartE2EDuration="2.123772523s" totalImagesPullingTime="368.161104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:07:29.12286197 +0000 UTC m=+2098.779552093" watchObservedRunningTime="2026-07-18 14:07:29.123772523 +0000 UTC m=+2098.780462652"
Jul 18 14:07:30.866055 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:30.866015    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:07:30.873096 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:30.873074    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:07:30.873380 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:30.873364    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:07:30.880105 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:30.880088    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:07:31.427172 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:31.427137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:35.117259 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:35.117219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:07:35.229415 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:35.229384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:07:38.152492 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:38.152455    2577 generic.go:356] "Generic (PLEG): container finished" podID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerID="ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092" exitCode=0
Jul 18 14:07:38.152945 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:38.152529    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" event={"ID":"7456631b-3875-4769-8a54-4bf85fadbc6e","Type":"ContainerDied","Data":"ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"}
Jul 18 14:07:38.153019 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:38.153000    2577 scope.go:117] "RemoveContainer" containerID="ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"
Jul 18 14:07:38.154022 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:38.153992    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:07:38.154098 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:38.154081    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:07:38.154139 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:38.154113    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:07:39.159730 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:39.159692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" event={"ID":"7456631b-3875-4769-8a54-4bf85fadbc6e","Type":"ContainerStarted","Data":"ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc"}
Jul 18 14:07:39.776449 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:39.776417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:39.777475 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:39.777455    2577 patch_prober.go:28] interesting pod/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 18 14:07:39.777569 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:39.777507    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 18 14:07:40.776916 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:40.776874    2577 patch_prober.go:28] interesting pod/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 18 14:07:40.777344 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:40.776934    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 18 14:07:41.090003 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:41.089898    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:41.776498 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:41.776454    2577 patch_prober.go:28] interesting pod/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 18 14:07:41.776695 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:41.776507    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 18 14:07:42.777229 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:42.777192    2577 patch_prober.go:28] interesting pod/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 18 14:07:42.777619 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:42.777250    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 18 14:07:43.780478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:43.780449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:07:43.781514 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:07:43.781494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:08:09.705940 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:09.705907    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"]
Jul 18 14:08:09.706469 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:09.706153    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" podUID="20a1623d-1e97-4179-8ddf-7ab17fc3e837" containerName="maas-api" containerID="cri-o://8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc" gracePeriod=30
Jul 18 14:08:09.732296 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:09.732274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:08:09.958254 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:09.958196    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:08:10.003781 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.003756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls\") pod \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") "
Jul 18 14:08:10.003907 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.003795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a1623d-1e97-4179-8ddf-7ab17fc3e837-openshift-service-ca\") pod \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") "
Jul 18 14:08:10.003907 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.003816    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82fk8\" (UniqueName: \"kubernetes.io/projected/20a1623d-1e97-4179-8ddf-7ab17fc3e837-kube-api-access-82fk8\") pod \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\" (UID: \"20a1623d-1e97-4179-8ddf-7ab17fc3e837\") "
Jul 18 14:08:10.004257 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.004232    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a1623d-1e97-4179-8ddf-7ab17fc3e837-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20a1623d-1e97-4179-8ddf-7ab17fc3e837" (UID: "20a1623d-1e97-4179-8ddf-7ab17fc3e837"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:08:10.005943 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.005911    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20a1623d-1e97-4179-8ddf-7ab17fc3e837" (UID: "20a1623d-1e97-4179-8ddf-7ab17fc3e837"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:08:10.006071 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.006030    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a1623d-1e97-4179-8ddf-7ab17fc3e837-kube-api-access-82fk8" (OuterVolumeSpecName: "kube-api-access-82fk8") pod "20a1623d-1e97-4179-8ddf-7ab17fc3e837" (UID: "20a1623d-1e97-4179-8ddf-7ab17fc3e837"). InnerVolumeSpecName "kube-api-access-82fk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:08:10.105342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.105313    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a1623d-1e97-4179-8ddf-7ab17fc3e837-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:08:10.105342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.105341    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a1623d-1e97-4179-8ddf-7ab17fc3e837-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:08:10.105342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.105355    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82fk8\" (UniqueName: \"kubernetes.io/projected/20a1623d-1e97-4179-8ddf-7ab17fc3e837-kube-api-access-82fk8\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:08:10.320169 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.320135    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a1623d-1e97-4179-8ddf-7ab17fc3e837" containerID="8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc" exitCode=0
Jul 18 14:08:10.320338 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.320179    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"
Jul 18 14:08:10.320338 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.320222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" event={"ID":"20a1623d-1e97-4179-8ddf-7ab17fc3e837","Type":"ContainerDied","Data":"8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc"}
Jul 18 14:08:10.320338 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.320271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq" event={"ID":"20a1623d-1e97-4179-8ddf-7ab17fc3e837","Type":"ContainerDied","Data":"400e134c0ea5591b2ae4db8a791701cdc22ff71c1367a440913a4dc19368717e"}
Jul 18 14:08:10.320338 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.320294    2577 scope.go:117] "RemoveContainer" containerID="8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc"
Jul 18 14:08:10.329081 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.329061    2577 scope.go:117] "RemoveContainer" containerID="8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc"
Jul 18 14:08:10.329321 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:08:10.329300    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc\": container with ID starting with 8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc not found: ID does not exist" containerID="8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc"
Jul 18 14:08:10.329394 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.329334    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc"} err="failed to get container status \"8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc\": rpc error: code = NotFound desc = could not find container \"8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc\": container with ID starting with 8efc77ba7a308cc4de5b270bac754bac67c863b179fcdbc7ae1d8fe32ac0fcdc not found: ID does not exist"
Jul 18 14:08:10.341330 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.341304    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"]
Jul 18 14:08:10.342894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.342876    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbfeb3b-67cf45467c-gq7nq"]
Jul 18 14:08:10.801651 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:10.801617    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a1623d-1e97-4179-8ddf-7ab17fc3e837" path="/var/lib/kubelet/pods/20a1623d-1e97-4179-8ddf-7ab17fc3e837/volumes"
Jul 18 14:08:14.415432 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:14.415386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:08:16.498261 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:16.498220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:08:21.531033 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:21.530995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:08:30.930378 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:30.930348    2577 scope.go:117] "RemoveContainer" containerID="f0a693560e283ff1811a72e553de630c12a0f55ece13eccbfed00e3613914f7a"
Jul 18 14:08:42.711092 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:42.711028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:08:58.183367 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:58.183337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:08:58.514833 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:08:58.514745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:03.928655 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:03.928615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:04.028735 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:04.028706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:04.169652 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:04.169611    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"]
Jul 18 14:09:04.169875 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:04.169854    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy" containerID="cri-o://ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc" gracePeriod=30
Jul 18 14:09:04.193547 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:04.193475    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:04.414465 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:04.414438    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:05.419702 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:05.419669    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:05.520007 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:05.519971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:05.620088 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:05.620058    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:06.297479 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:06.297440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:06.397199 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:06.397153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:06.497540 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:06.497506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:07.789088 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:07.789054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:09.417925 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.417896    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:09:09.589721 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.589696    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-socket\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.589865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.589724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-token\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.589865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.589742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-certs\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.589865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.589776    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-data\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.589865 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.589797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-podinfo\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.590117 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.589926    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-credential-socket\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.590117 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590008    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7456631b-3875-4769-8a54-4bf85fadbc6e-istiod-ca-cert\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.590117 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590009    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:09:09.590117 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590016    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:09:09.590117 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590113    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl862\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-kube-api-access-nl862\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.590382 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590143    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-envoy\") pod \"7456631b-3875-4769-8a54-4bf85fadbc6e\" (UID: \"7456631b-3875-4769-8a54-4bf85fadbc6e\") "
Jul 18 14:09:09.590382 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590238    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:09:09.590382 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590340    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7456631b-3875-4769-8a54-4bf85fadbc6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:09:09.590519 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590400    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.590519 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590421    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-workload-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.590519 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590435    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-credential-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.590835 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.590807    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:09:09.592002 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.591967    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:09:09.592287 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.592253    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-kube-api-access-nl862" (OuterVolumeSpecName: "kube-api-access-nl862") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "kube-api-access-nl862". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:09:09.592377 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.592340    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 14:09:09.592377 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.592353    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7456631b-3875-4769-8a54-4bf85fadbc6e" (UID: "7456631b-3875-4769-8a54-4bf85fadbc6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:09:09.610794 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.610768    2577 generic.go:356] "Generic (PLEG): container finished" podID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerID="ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc" exitCode=0
Jul 18 14:09:09.610878 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.610810    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"
Jul 18 14:09:09.610878 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.610841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" event={"ID":"7456631b-3875-4769-8a54-4bf85fadbc6e","Type":"ContainerDied","Data":"ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc"}
Jul 18 14:09:09.610972 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.610910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf" event={"ID":"7456631b-3875-4769-8a54-4bf85fadbc6e","Type":"ContainerDied","Data":"0e5af10bb1c264d0a5907ba7419dea7747b8acd4b815fbbd18cc70a32d6cac85"}
Jul 18 14:09:09.610972 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.610939    2577 scope.go:117] "RemoveContainer" containerID="ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc"
Jul 18 14:09:09.620364 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.620352    2577 scope.go:117] "RemoveContainer" containerID="ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"
Jul 18 14:09:09.629739 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.629726    2577 scope.go:117] "RemoveContainer" containerID="ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc"
Jul 18 14:09:09.630417 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:09:09.630329    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc\": container with ID starting with ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc not found: ID does not exist" containerID="ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc"
Jul 18 14:09:09.630417 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.630369    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc"} err="failed to get container status \"ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc\": rpc error: code = NotFound desc = could not find container \"ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc\": container with ID starting with ca6e345d5008c9dbf1f7e42f10c3c5512d356578b7c36ffecf83cc02121142cc not found: ID does not exist"
Jul 18 14:09:09.630417 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.630392    2577 scope.go:117] "RemoveContainer" containerID="ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"
Jul 18 14:09:09.631250 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:09:09.631145    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092\": container with ID starting with ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092 not found: ID does not exist" containerID="ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"
Jul 18 14:09:09.631250 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.631175    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092"} err="failed to get container status \"ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092\": rpc error: code = NotFound desc = could not find container \"ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092\": container with ID starting with ceec22b008c53203104282b9a04bb315d4eb0bbd89ea19e8c7b95637d1784092 not found: ID does not exist"
Jul 18 14:09:09.632870 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.632848    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"]
Jul 18 14:09:09.636131 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.636102    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ffbfeb3b-openshift-default-54d4c48fdc-2sfkf"]
Jul 18 14:09:09.691451 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.691428    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl862\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-kube-api-access-nl862\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.691451 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.691450    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-envoy\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.691586 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.691463    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.691586 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.691472    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-data\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.691586 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.691479    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7456631b-3875-4769-8a54-4bf85fadbc6e-istio-podinfo\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:09.691586 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:09.691487    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7456631b-3875-4769-8a54-4bf85fadbc6e-istiod-ca-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:09:10.802709 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:10.802679    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" path="/var/lib/kubelet/pods/7456631b-3875-4769-8a54-4bf85fadbc6e/volumes"
Jul 18 14:09:13.482380 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:13.482342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:13.581631 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:13.581601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:14.132879 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:14.132850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:19.960804 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:19.960768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:20.066474 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:20.066439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:20.163791 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:20.163755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:30.632695 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:30.632661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:30.948245 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:30.948161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:30.958004 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:30.957982    2577 scope.go:117] "RemoveContainer" containerID="3938f61e7ac00672fe718c8cb69112186d1fe1e43dff2c2039426bd6ab4a7861"
Jul 18 14:09:51.889898 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:51.889865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:52.752719 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:52.752641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:52.850098 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:52.850066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:57.998983 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:57.998950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:58.098908 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.098883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:58.325820 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.325793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:58.604953 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.604873    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"]
Jul 18 14:09:58.605282 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605265    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy"
Jul 18 14:09:58.605282 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605281    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy"
Jul 18 14:09:58.605429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605300    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a1623d-1e97-4179-8ddf-7ab17fc3e837" containerName="maas-api"
Jul 18 14:09:58.605429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605307    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a1623d-1e97-4179-8ddf-7ab17fc3e837" containerName="maas-api"
Jul 18 14:09:58.605429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605363    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy"
Jul 18 14:09:58.605429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605371    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy"
Jul 18 14:09:58.605429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.605379    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a1623d-1e97-4179-8ddf-7ab17fc3e837" containerName="maas-api"
Jul 18 14:09:58.608124 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.608107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.610545 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.610526    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e5f2a1-openshift-default-dockercfg-kfcm9\""
Jul 18 14:09:58.624153 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.624131    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"]
Jul 18 14:09:58.626926 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.626908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:58.693990 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.693949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-token\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694184 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-socket\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694184 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-credential-socket\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694305 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-certs\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694305 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-envoy\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694408 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxr95\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-kube-api-access-mxr95\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694408 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-data\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694408 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-podinfo\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.694560 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.694407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istiod-ca-cert\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.795808 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-certs\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.795969 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-envoy\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.795969 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxr95\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-kube-api-access-mxr95\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.795969 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-data\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.795969 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-podinfo\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.795969 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istiod-ca-cert\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796279 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.795986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-token\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796279 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.796012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-socket\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796279 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.796066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-credential-socket\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796446 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.796294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-certs\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796446 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.796338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-credential-socket\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796557 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.796498    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-data\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.796656 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.796633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-socket\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.797076 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.797058    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istiod-ca-cert\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.798597 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.798577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-envoy\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.799023 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.799006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-podinfo\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.803680 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.803657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxr95\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-kube-api-access-mxr95\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.803851 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.803832    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-token\") pod \"e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") " pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.919452 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.919386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:09:58.947212 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:58.947178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:09:59.054200 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.054173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"]
Jul 18 14:09:59.055786 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:09:59.055760    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaabc2492_1f26_4343_a8cf_01e6c0cb97aa.slice/crio-fd2c1cdcf83b1552e43c57aa8e1a52eb46423e234c5c008ce9716c4783054e4c WatchSource:0}: Error finding container fd2c1cdcf83b1552e43c57aa8e1a52eb46423e234c5c008ce9716c4783054e4c: Status 404 returned error can't find the container with id fd2c1cdcf83b1552e43c57aa8e1a52eb46423e234c5c008ce9716c4783054e4c
Jul 18 14:09:59.057993 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.057957    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:09:59.058128 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.058019    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:09:59.058128 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.058066    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:09:59.848307 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.848273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" event={"ID":"aabc2492-1f26-4343-a8cf-01e6c0cb97aa","Type":"ContainerStarted","Data":"212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"}
Jul 18 14:09:59.848494 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.848323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" event={"ID":"aabc2492-1f26-4343-a8cf-01e6c0cb97aa","Type":"ContainerStarted","Data":"fd2c1cdcf83b1552e43c57aa8e1a52eb46423e234c5c008ce9716c4783054e4c"}
Jul 18 14:09:59.866735 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.866691    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" podStartSLOduration=1.866677355 podStartE2EDuration="1.866677355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:09:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:09:59.865367387 +0000 UTC m=+2249.522057516" watchObservedRunningTime="2026-07-18 14:09:59.866677355 +0000 UTC m=+2249.523367484"
Jul 18 14:09:59.919729 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:09:59.919708    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:00.925137 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:00.925102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:01.738338 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.738300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:01.829495 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.829459    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"]
Jul 18 14:10:01.830068 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.830023    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy"
Jul 18 14:10:01.830068 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.830065    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7456631b-3875-4769-8a54-4bf85fadbc6e" containerName="istio-proxy"
Jul 18 14:10:01.830417 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.830395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:01.833644 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.833624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:01.836127 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.836102    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e5f2a1-serving-cert\""
Jul 18 14:10:01.836244 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.836159    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-97gtb\""
Jul 18 14:10:01.843628 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.843602    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"]
Jul 18 14:10:01.858252 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.858228    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:01.859139 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.859122    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:01.990874 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:01.990790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:02.023309 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.023267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.023524 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.023317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d573e64-f647-4116-b308-58c1897fea9f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.023524 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.023345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkpmg\" (UniqueName: \"kubernetes.io/projected/2d573e64-f647-4116-b308-58c1897fea9f-kube-api-access-hkpmg\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.124026 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.123993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.124232 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.124027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d573e64-f647-4116-b308-58c1897fea9f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.124232 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.124075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkpmg\" (UniqueName: \"kubernetes.io/projected/2d573e64-f647-4116-b308-58c1897fea9f-kube-api-access-hkpmg\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.124232 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:10:02.124138    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-serving-cert: secret "maas-api-e2e-inf-a-e5f2a1-serving-cert" not found
Jul 18 14:10:02.124232 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:10:02.124199    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls podName:2d573e64-f647-4116-b308-58c1897fea9f nodeName:}" failed. No retries permitted until 2026-07-18 14:10:02.624183423 +0000 UTC m=+2252.280873531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls") pod "maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" (UID: "2d573e64-f647-4116-b308-58c1897fea9f") : secret "maas-api-e2e-inf-a-e5f2a1-serving-cert" not found
Jul 18 14:10:02.124772 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.124748    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d573e64-f647-4116-b308-58c1897fea9f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.132285 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.132261    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkpmg\" (UniqueName: \"kubernetes.io/projected/2d573e64-f647-4116-b308-58c1897fea9f-kube-api-access-hkpmg\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.380840 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.380811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:02.484013 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.483979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:02.628784 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.628751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:02.628964 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:10:02.628914    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-serving-cert: secret "maas-api-e2e-inf-a-e5f2a1-serving-cert" not found
Jul 18 14:10:02.629007 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:10:02.628973    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls podName:2d573e64-f647-4116-b308-58c1897fea9f nodeName:}" failed. No retries permitted until 2026-07-18 14:10:03.628958708 +0000 UTC m=+2253.285648816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls") pod "maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" (UID: "2d573e64-f647-4116-b308-58c1897fea9f") : secret "maas-api-e2e-inf-a-e5f2a1-serving-cert" not found
Jul 18 14:10:02.780076 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:02.780034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:03.639433 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:03.639394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:03.641711 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:03.641692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:03.648696 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:03.648661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:03.781662 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:03.779868    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"]
Jul 18 14:10:03.784500 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:10:03.784474    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d573e64_f647_4116_b308_58c1897fea9f.slice/crio-8823b32781999b9e38600b9dd20c41d4b1904e60560362ded7c7acbf07bcff96 WatchSource:0}: Error finding container 8823b32781999b9e38600b9dd20c41d4b1904e60560362ded7c7acbf07bcff96: Status 404 returned error can't find the container with id 8823b32781999b9e38600b9dd20c41d4b1904e60560362ded7c7acbf07bcff96
Jul 18 14:10:03.873657 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:03.873625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" event={"ID":"2d573e64-f647-4116-b308-58c1897fea9f","Type":"ContainerStarted","Data":"8823b32781999b9e38600b9dd20c41d4b1904e60560362ded7c7acbf07bcff96"}
Jul 18 14:10:05.888178 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:05.888143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" event={"ID":"2d573e64-f647-4116-b308-58c1897fea9f","Type":"ContainerStarted","Data":"c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826"}
Jul 18 14:10:05.888533 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:05.888200    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:05.906796 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:05.906751    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" podStartSLOduration=3.870588992 podStartE2EDuration="4.906736006s" totalImagesPullingTime="1.036147014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:10:05.90501896 +0000 UTC m=+2255.561709091" watchObservedRunningTime="2026-07-18 14:10:05.906736006 +0000 UTC m=+2255.563426135"
Jul 18 14:10:06.266883 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:06.266799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:06.561212 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:06.561180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:06.843058 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:06.842952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:07.389583 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:07.389547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:09.486093 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.486058    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:09.576332 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.576287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:09.794110 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.794073    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"]
Jul 18 14:10:09.814455 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.814425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.821059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.821006    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-45d555-serving-cert\""
Jul 18 14:10:09.824198 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.824159    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"]
Jul 18 14:10:09.874880 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.874843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:09.887606 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.887574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-maas-api-tls\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.887897 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.887870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-openshift-service-ca\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.888023 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.888007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wrzl\" (UniqueName: \"kubernetes.io/projected/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-kube-api-access-5wrzl\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.959248 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.959223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:09.988749 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.988714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-maas-api-tls\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.989084 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.989062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-openshift-service-ca\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.989995 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.989978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wrzl\" (UniqueName: \"kubernetes.io/projected/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-kube-api-access-5wrzl\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:09.990236 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:09.989922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-openshift-service-ca\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:10.004059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.000863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-maas-api-tls\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:10.004059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.004023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wrzl\" (UniqueName: \"kubernetes.io/projected/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-kube-api-access-5wrzl\") pod \"maas-api-e2e-inf-b-45d555-776c598fd8-85knp\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:10.131252 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.131156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:10.312578 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.312525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"]
Jul 18 14:10:10.318897 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:10:10.318824    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8788dc1a_3dbc_4c1a_8012_28c0d834fcca.slice/crio-8c3385b31efe1c1d14aad0b1b695ab2c26572b90221b8fdee35dde62cb5673b2 WatchSource:0}: Error finding container 8c3385b31efe1c1d14aad0b1b695ab2c26572b90221b8fdee35dde62cb5673b2: Status 404 returned error can't find the container with id 8c3385b31efe1c1d14aad0b1b695ab2c26572b90221b8fdee35dde62cb5673b2
Jul 18 14:10:10.349926 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.349889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:10.451086 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.450956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:10.922939 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.922895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp" event={"ID":"8788dc1a-3dbc-4c1a-8012-28c0d834fcca","Type":"ContainerStarted","Data":"12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce"}
Jul 18 14:10:10.923346 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.922953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp" event={"ID":"8788dc1a-3dbc-4c1a-8012-28c0d834fcca","Type":"ContainerStarted","Data":"8c3385b31efe1c1d14aad0b1b695ab2c26572b90221b8fdee35dde62cb5673b2"}
Jul 18 14:10:10.923346 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.923008    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:10.939188 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:10.939144    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp" podStartSLOduration=1.659677071 podStartE2EDuration="1.939130373s" totalImagesPullingTime="279.453302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:10:10.937695329 +0000 UTC m=+2260.594385459" watchObservedRunningTime="2026-07-18 14:10:10.939130373 +0000 UTC m=+2260.595820502"
Jul 18 14:10:11.897866 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:11.897835    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:10:13.938722 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:13.938693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:14.944061 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:14.943968    2577 generic.go:356] "Generic (PLEG): container finished" podID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerID="212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6" exitCode=0
Jul 18 14:10:14.944472 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:14.944064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" event={"ID":"aabc2492-1f26-4343-a8cf-01e6c0cb97aa","Type":"ContainerDied","Data":"212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"}
Jul 18 14:10:14.944472 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:14.944457    2577 scope.go:117] "RemoveContainer" containerID="212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"
Jul 18 14:10:14.945472 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:14.945443    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:10:14.945544 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:14.945527    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:10:14.945605 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:14.945591    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:10:15.951110 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:15.951071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" event={"ID":"aabc2492-1f26-4343-a8cf-01e6c0cb97aa","Type":"ContainerStarted","Data":"8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02"}
Jul 18 14:10:16.859119 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:16.859084    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:16.919741 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:16.919707    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:16.920914 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:16.920893    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 18 14:10:16.920973 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:16.920954    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 18 14:10:16.933647 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:16.933614    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:10:17.920226 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:17.920189    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 18 14:10:17.920603 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:17.920259    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 18 14:10:18.920025 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:18.919990    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 18 14:10:18.920233 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:18.920067    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 18 14:10:19.924171 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:19.924140    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:19.925187 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:19.925162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:10:23.081516 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:23.081478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:23.184290 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:23.184261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:24.206562 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:24.206524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:30.982951 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:30.982920    2577 scope.go:117] "RemoveContainer" containerID="f69e9f81e63f07331ab431afe03c29beca2580f55c51842a5c8478b90fb4692f"
Jul 18 14:10:31.197783 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:31.197738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:31.301373 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:31.301341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:10:49.712004 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:10:49.711966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:07.048182 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:07.048134    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:08.036744 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:08.036713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:08.733129 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:08.733095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:08.833304 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:08.833264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:18.523765 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.523726    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"]
Jul 18 14:11:18.524272 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.523933    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" podUID="2d573e64-f647-4116-b308-58c1897fea9f" containerName="maas-api" containerID="cri-o://c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826" gracePeriod=30
Jul 18 14:11:18.548052 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.548016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:18.776122 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.776060    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:11:18.915684 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.915649    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d573e64-f647-4116-b308-58c1897fea9f-openshift-service-ca\") pod \"2d573e64-f647-4116-b308-58c1897fea9f\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") "
Jul 18 14:11:18.915848 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.915694    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") pod \"2d573e64-f647-4116-b308-58c1897fea9f\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") "
Jul 18 14:11:18.915848 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.915824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkpmg\" (UniqueName: \"kubernetes.io/projected/2d573e64-f647-4116-b308-58c1897fea9f-kube-api-access-hkpmg\") pod \"2d573e64-f647-4116-b308-58c1897fea9f\" (UID: \"2d573e64-f647-4116-b308-58c1897fea9f\") "
Jul 18 14:11:18.916132 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.916109    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d573e64-f647-4116-b308-58c1897fea9f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d573e64-f647-4116-b308-58c1897fea9f" (UID: "2d573e64-f647-4116-b308-58c1897fea9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:11:18.917816 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.917790    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d573e64-f647-4116-b308-58c1897fea9f" (UID: "2d573e64-f647-4116-b308-58c1897fea9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:11:18.917909 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:18.917891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d573e64-f647-4116-b308-58c1897fea9f-kube-api-access-hkpmg" (OuterVolumeSpecName: "kube-api-access-hkpmg") pod "2d573e64-f647-4116-b308-58c1897fea9f" (UID: "2d573e64-f647-4116-b308-58c1897fea9f"). InnerVolumeSpecName "kube-api-access-hkpmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:11:19.017265 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.017237    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d573e64-f647-4116-b308-58c1897fea9f-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:19.017265 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.017262    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d573e64-f647-4116-b308-58c1897fea9f-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:19.017265 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.017272    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkpmg\" (UniqueName: \"kubernetes.io/projected/2d573e64-f647-4116-b308-58c1897fea9f-kube-api-access-hkpmg\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:19.264480 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.264390    2577 generic.go:356] "Generic (PLEG): container finished" podID="2d573e64-f647-4116-b308-58c1897fea9f" containerID="c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826" exitCode=0
Jul 18 14:11:19.264480 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.264439    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"
Jul 18 14:11:19.264649 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.264475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" event={"ID":"2d573e64-f647-4116-b308-58c1897fea9f","Type":"ContainerDied","Data":"c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826"}
Jul 18 14:11:19.264649 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.264535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs" event={"ID":"2d573e64-f647-4116-b308-58c1897fea9f","Type":"ContainerDied","Data":"8823b32781999b9e38600b9dd20c41d4b1904e60560362ded7c7acbf07bcff96"}
Jul 18 14:11:19.264649 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.264556    2577 scope.go:117] "RemoveContainer" containerID="c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826"
Jul 18 14:11:19.276339 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.276315    2577 scope.go:117] "RemoveContainer" containerID="c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826"
Jul 18 14:11:19.276656 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:11:19.276631    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826\": container with ID starting with c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826 not found: ID does not exist" containerID="c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826"
Jul 18 14:11:19.276745 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.276667    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826"} err="failed to get container status \"c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826\": rpc error: code = NotFound desc = could not find container \"c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826\": container with ID starting with c13016e062e4e554aa3124fefdf45966dbcba2c416010ebc404361b29ea85826 not found: ID does not exist"
Jul 18 14:11:19.289122 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.289096    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"]
Jul 18 14:11:19.291787 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:19.291765    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5f2a1-849db9f589-jhmbs"]
Jul 18 14:11:20.802146 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:20.802117    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d573e64-f647-4116-b308-58c1897fea9f" path="/var/lib/kubelet/pods/2d573e64-f647-4116-b308-58c1897fea9f/volumes"
Jul 18 14:11:22.321694 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:22.321635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:22.374004 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:22.373977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:23.211521 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:23.211484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:23.589459 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:23.589425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:29.357327 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:29.357293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:29.460433 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:29.460401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:29.603623 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:29.603590    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"]
Jul 18 14:11:29.603900 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:29.603876    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy" containerID="cri-o://8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02" gracePeriod=30
Jul 18 14:11:29.620365 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:29.620299    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:29.843327 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:29.843293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:30.438857 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:30.438824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:30.584919 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:30.584880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:31.002245 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:31.002218    2577 scope.go:117] "RemoveContainer" containerID="d5309bfd7177bece7a177ecf011fcf72c8bc341f963f456bcac5cd433be5ec63"
Jul 18 14:11:34.848617 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.848593    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:11:34.959552 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-data\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959552 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959514    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-certs\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959552 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959536    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-podinfo\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istiod-ca-cert\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-socket\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959673    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-envoy\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-credential-socket\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxr95\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-kube-api-access-mxr95\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959752    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-token\") pod \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\" (UID: \"aabc2492-1f26-4343-a8cf-01e6c0cb97aa\") "
Jul 18 14:11:34.959822 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:34.960227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.959992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:34.960227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.960012    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:34.960227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.960118    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:34.960227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.960118    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:11:34.960227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.960136    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-workload-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:34.960227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.960181    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-credential-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:34.960744 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.960725    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:34.961878 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.961856    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:11:34.961967 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.961896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-kube-api-access-mxr95" (OuterVolumeSpecName: "kube-api-access-mxr95") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "kube-api-access-mxr95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:11:34.962197 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.962172    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 14:11:34.962250 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:34.962198    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "aabc2492-1f26-4343-a8cf-01e6c0cb97aa" (UID: "aabc2492-1f26-4343-a8cf-01e6c0cb97aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:11:35.061405 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.061367    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-data\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:35.061405 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.061399    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-podinfo\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:35.061405 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.061409    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istiod-ca-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:35.061405 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.061417    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-envoy\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:35.061638 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.061425    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxr95\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-kube-api-access-mxr95\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:35.061638 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.061432    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aabc2492-1f26-4343-a8cf-01e6c0cb97aa-istio-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:35.348391 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.348352    2577 generic.go:356] "Generic (PLEG): container finished" podID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerID="8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02" exitCode=0
Jul 18 14:11:35.348574 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.348399    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"
Jul 18 14:11:35.348574 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.348439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" event={"ID":"aabc2492-1f26-4343-a8cf-01e6c0cb97aa","Type":"ContainerDied","Data":"8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02"}
Jul 18 14:11:35.348574 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.348499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx" event={"ID":"aabc2492-1f26-4343-a8cf-01e6c0cb97aa","Type":"ContainerDied","Data":"fd2c1cdcf83b1552e43c57aa8e1a52eb46423e234c5c008ce9716c4783054e4c"}
Jul 18 14:11:35.348574 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.348521    2577 scope.go:117] "RemoveContainer" containerID="8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02"
Jul 18 14:11:35.357246 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.357228    2577 scope.go:117] "RemoveContainer" containerID="212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"
Jul 18 14:11:35.364713 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.364696    2577 scope.go:117] "RemoveContainer" containerID="8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02"
Jul 18 14:11:35.364939 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:11:35.364919    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02\": container with ID starting with 8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02 not found: ID does not exist" containerID="8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02"
Jul 18 14:11:35.364988 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.364948    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02"} err="failed to get container status \"8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02\": rpc error: code = NotFound desc = could not find container \"8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02\": container with ID starting with 8218eddcda2636fecdaa5fe250ad10268c48ba57d05516acd5ab5fe424fe3f02 not found: ID does not exist"
Jul 18 14:11:35.364988 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.364965    2577 scope.go:117] "RemoveContainer" containerID="212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"
Jul 18 14:11:35.365211 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:11:35.365194    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6\": container with ID starting with 212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6 not found: ID does not exist" containerID="212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"
Jul 18 14:11:35.365275 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.365217    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6"} err="failed to get container status \"212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6\": rpc error: code = NotFound desc = could not find container \"212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6\": container with ID starting with 212a9ff84145fa734740e3b506c4814b50fb13ddcd0235c7c04c499e3848d1c6 not found: ID does not exist"
Jul 18 14:11:35.368891 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.368869    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"]
Jul 18 14:11:35.375050 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:35.375015    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e5f2a1-openshift-default-cd54f54df-tw4mx"]
Jul 18 14:11:36.802377 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:36.802347    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" path="/var/lib/kubelet/pods/aabc2492-1f26-4343-a8cf-01e6c0cb97aa/volumes"
Jul 18 14:11:40.309436 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:40.309401    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"]
Jul 18 14:11:40.310211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:40.310176    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp" podUID="8788dc1a-3dbc-4c1a-8012-28c0d834fcca" containerName="maas-api" containerID="cri-o://12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce" gracePeriod=30
Jul 18 14:11:40.340416 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:40.340386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:41.553308 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.553283    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:11:41.724867 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.724784    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-maas-api-tls\") pod \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") "
Jul 18 14:11:41.724867 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.724824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wrzl\" (UniqueName: \"kubernetes.io/projected/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-kube-api-access-5wrzl\") pod \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") "
Jul 18 14:11:41.724867 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.724864    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-openshift-service-ca\") pod \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\" (UID: \"8788dc1a-3dbc-4c1a-8012-28c0d834fcca\") "
Jul 18 14:11:41.725318 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.725293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8788dc1a-3dbc-4c1a-8012-28c0d834fcca" (UID: "8788dc1a-3dbc-4c1a-8012-28c0d834fcca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:11:41.726856 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.726837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-kube-api-access-5wrzl" (OuterVolumeSpecName: "kube-api-access-5wrzl") pod "8788dc1a-3dbc-4c1a-8012-28c0d834fcca" (UID: "8788dc1a-3dbc-4c1a-8012-28c0d834fcca"). InnerVolumeSpecName "kube-api-access-5wrzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:11:41.726927 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.726851    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8788dc1a-3dbc-4c1a-8012-28c0d834fcca" (UID: "8788dc1a-3dbc-4c1a-8012-28c0d834fcca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:11:41.832655 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.826051    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:41.832655 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.826088    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:41.832655 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:41.826110    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wrzl\" (UniqueName: \"kubernetes.io/projected/8788dc1a-3dbc-4c1a-8012-28c0d834fcca-kube-api-access-5wrzl\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:11:42.381650 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.381613    2577 generic.go:356] "Generic (PLEG): container finished" podID="8788dc1a-3dbc-4c1a-8012-28c0d834fcca" containerID="12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce" exitCode=0
Jul 18 14:11:42.381795 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.381681    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"
Jul 18 14:11:42.381795 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.381697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp" event={"ID":"8788dc1a-3dbc-4c1a-8012-28c0d834fcca","Type":"ContainerDied","Data":"12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce"}
Jul 18 14:11:42.381795 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.381780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp" event={"ID":"8788dc1a-3dbc-4c1a-8012-28c0d834fcca","Type":"ContainerDied","Data":"8c3385b31efe1c1d14aad0b1b695ab2c26572b90221b8fdee35dde62cb5673b2"}
Jul 18 14:11:42.381907 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.381803    2577 scope.go:117] "RemoveContainer" containerID="12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce"
Jul 18 14:11:42.390051 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.390014    2577 scope.go:117] "RemoveContainer" containerID="12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce"
Jul 18 14:11:42.390280 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:11:42.390263    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce\": container with ID starting with 12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce not found: ID does not exist" containerID="12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce"
Jul 18 14:11:42.390326 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.390288    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce"} err="failed to get container status \"12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce\": rpc error: code = NotFound desc = could not find container \"12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce\": container with ID starting with 12bf9c18af732417e2e1a6c810874d1e54de4397de900b661b9a2c04adbad7ce not found: ID does not exist"
Jul 18 14:11:42.401277 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.401251    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"]
Jul 18 14:11:42.404816 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.404796    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-45d555-776c598fd8-85knp"]
Jul 18 14:11:42.802246 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:42.802209    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8788dc1a-3dbc-4c1a-8012-28c0d834fcca" path="/var/lib/kubelet/pods/8788dc1a-3dbc-4c1a-8012-28c0d834fcca/volumes"
Jul 18 14:11:45.017921 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:45.017885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:45.375882 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:45.375841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:50.956519 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:50.956485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:51.056310 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:51.056278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:51.215125 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:51.214999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:51.429917 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:51.429871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:51.754859 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:51.754828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:52.019913 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.019825    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"]
Jul 18 14:11:52.020297 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020284    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d573e64-f647-4116-b308-58c1897fea9f" containerName="maas-api"
Jul 18 14:11:52.020343 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020304    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d573e64-f647-4116-b308-58c1897fea9f" containerName="maas-api"
Jul 18 14:11:52.020343 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020326    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8788dc1a-3dbc-4c1a-8012-28c0d834fcca" containerName="maas-api"
Jul 18 14:11:52.020343 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020335    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8788dc1a-3dbc-4c1a-8012-28c0d834fcca" containerName="maas-api"
Jul 18 14:11:52.020478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020348    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy"
Jul 18 14:11:52.020478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020356    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy"
Jul 18 14:11:52.020478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020373    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy"
Jul 18 14:11:52.020478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020382    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy"
Jul 18 14:11:52.020478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020469    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy"
Jul 18 14:11:52.020478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020479    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d573e64-f647-4116-b308-58c1897fea9f" containerName="maas-api"
Jul 18 14:11:52.020677 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.020491    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8788dc1a-3dbc-4c1a-8012-28c0d834fcca" containerName="maas-api"
Jul 18 14:11:52.025098 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.025078    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.027157 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.027138    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-31be2e-openshift-default-dockercfg-n57gz\""
Jul 18 14:11:52.033147 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.033126    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"]
Jul 18 14:11:52.048429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.048400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:52.100563 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-certs\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100711 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-envoy\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100711 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istiod-ca-cert\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100711 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xllhd\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-kube-api-access-xllhd\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-data\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-credential-socket\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-token\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-socket\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.100894 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.100873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-podinfo\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.201695 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.201660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-certs\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.201893 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.201714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-envoy\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.201893 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.201742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istiod-ca-cert\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.201893 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.201777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xllhd\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-kube-api-access-xllhd\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202076 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.201919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-data\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202433 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-certs\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202540 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-credential-socket\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202540 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-token\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202540 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-data\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202697 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-socket\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.202697 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-podinfo\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.206059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-credential-socket\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.206059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.202976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istiod-ca-cert\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.206059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.203091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-socket\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.208904 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.208879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-envoy\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.209156 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.209134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-podinfo\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.215631 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.215607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-token\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.215834 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.215816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xllhd\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-kube-api-access-xllhd\") pod \"e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") " pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.281131 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.281098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:52.336018 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.335983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:52.468323 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.468300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"]
Jul 18 14:11:52.469946 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:11:52.469921    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f12abdd_8c92_4c5d_ab86_1e77e1a203a4.slice/crio-0f490c2a646350dfaa6be66431ceaa25f95a5af9302a1488543b871a2aa5d20c WatchSource:0}: Error finding container 0f490c2a646350dfaa6be66431ceaa25f95a5af9302a1488543b871a2aa5d20c: Status 404 returned error can't find the container with id 0f490c2a646350dfaa6be66431ceaa25f95a5af9302a1488543b871a2aa5d20c
Jul 18 14:11:52.471965 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.471932    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:11:52.472076 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.471997    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:11:52.472076 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:52.472030    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:11:53.434428 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:53.434394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" event={"ID":"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4","Type":"ContainerStarted","Data":"c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"}
Jul 18 14:11:53.434817 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:53.434448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" event={"ID":"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4","Type":"ContainerStarted","Data":"0f490c2a646350dfaa6be66431ceaa25f95a5af9302a1488543b871a2aa5d20c"}
Jul 18 14:11:53.454320 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:53.454274    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podStartSLOduration=2.454259235 podStartE2EDuration="2.454259235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:11:53.452813272 +0000 UTC m=+2363.109503401" watchObservedRunningTime="2026-07-18 14:11:53.454259235 +0000 UTC m=+2363.110949766"
Jul 18 14:11:54.336922 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:54.336883    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:54.341237 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:54.341212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:54.439405 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:54.439372    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:54.440196 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:54.440175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:11:55.330180 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.330153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:55.428246 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.428213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:55.449885 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.449847    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"]
Jul 18 14:11:55.450459 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.450438    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aabc2492-1f26-4343-a8cf-01e6c0cb97aa" containerName="istio-proxy"
Jul 18 14:11:55.453892 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.453868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.455971 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.455951    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-31be2e-serving-cert\""
Jul 18 14:11:55.456103 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.456064    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-97gtb\""
Jul 18 14:11:55.462841 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.462807    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"]
Jul 18 14:11:55.490447 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.490398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:55.531130 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.531099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.531537 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.531320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a017a9e9-183b-48da-805c-ecfa13b9de40-openshift-service-ca\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.531537 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.531357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cf7d\" (UniqueName: \"kubernetes.io/projected/a017a9e9-183b-48da-805c-ecfa13b9de40-kube-api-access-6cf7d\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.632417 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.632335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a017a9e9-183b-48da-805c-ecfa13b9de40-openshift-service-ca\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.632417 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.632371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cf7d\" (UniqueName: \"kubernetes.io/projected/a017a9e9-183b-48da-805c-ecfa13b9de40-kube-api-access-6cf7d\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.632643 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.632430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.632643 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:11:55.632595    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-serving-cert: secret "maas-api-e2e-api-a-31be2e-serving-cert" not found
Jul 18 14:11:55.632752 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:11:55.632674    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls podName:a017a9e9-183b-48da-805c-ecfa13b9de40 nodeName:}" failed. No retries permitted until 2026-07-18 14:11:56.132651515 +0000 UTC m=+2365.789341627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls") pod "maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" (UID: "a017a9e9-183b-48da-805c-ecfa13b9de40") : secret "maas-api-e2e-api-a-31be2e-serving-cert" not found
Jul 18 14:11:55.633072 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.633027    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a017a9e9-183b-48da-805c-ecfa13b9de40-openshift-service-ca\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.640600 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.640578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cf7d\" (UniqueName: \"kubernetes.io/projected/a017a9e9-183b-48da-805c-ecfa13b9de40-kube-api-access-6cf7d\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:55.688617 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:55.688593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:56.102451 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:56.102413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:56.137562 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:56.137529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:56.139786 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:56.139770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls\") pod \"maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:56.366954 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:56.366870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:56.498734 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:56.498701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"]
Jul 18 14:11:56.501841 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:11:56.501814    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda017a9e9_183b_48da_805c_ecfa13b9de40.slice/crio-a0816eb54c9849e015b6d75505a1ee2a551c204788f088672d476055f140e05a WatchSource:0}: Error finding container a0816eb54c9849e015b6d75505a1ee2a551c204788f088672d476055f140e05a: Status 404 returned error can't find the container with id a0816eb54c9849e015b6d75505a1ee2a551c204788f088672d476055f140e05a
Jul 18 14:11:57.287940 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:57.287909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:11:57.458824 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:57.458785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" event={"ID":"a017a9e9-183b-48da-805c-ecfa13b9de40","Type":"ContainerStarted","Data":"bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd"}
Jul 18 14:11:57.459000 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:57.458847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" event={"ID":"a017a9e9-183b-48da-805c-ecfa13b9de40","Type":"ContainerStarted","Data":"a0816eb54c9849e015b6d75505a1ee2a551c204788f088672d476055f140e05a"}
Jul 18 14:11:57.459000 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:57.458893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:11:57.475761 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:57.475716    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" podStartSLOduration=2.046443737 podStartE2EDuration="2.475701793s" totalImagesPullingTime="429.258056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:11:57.474562398 +0000 UTC m=+2367.131252526" watchObservedRunningTime="2026-07-18 14:11:57.475701793 +0000 UTC m=+2367.132391921"
Jul 18 14:11:59.802920 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:11:59.802890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:00.238500 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:00.238421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:00.519935 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:00.519849    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:00.893697 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:00.893661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:03.133408 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.133370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:03.217011 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.216982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:03.222312 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.222284    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"]
Jul 18 14:12:03.226207 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.226184    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.228743 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.228722    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6ee9e9-serving-cert\""
Jul 18 14:12:03.236106 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.236088    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"]
Jul 18 14:12:03.301734 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.301701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1407c308-c2b5-4da0-b79a-78ffd4efaf17-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.301902 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.301822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79cnb\" (UniqueName: \"kubernetes.io/projected/1407c308-c2b5-4da0-b79a-78ffd4efaf17-kube-api-access-79cnb\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.301902 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.301884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1407c308-c2b5-4da0-b79a-78ffd4efaf17-maas-api-tls\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.309817 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.309795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:03.403285 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.403195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1407c308-c2b5-4da0-b79a-78ffd4efaf17-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.403444 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.403309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79cnb\" (UniqueName: \"kubernetes.io/projected/1407c308-c2b5-4da0-b79a-78ffd4efaf17-kube-api-access-79cnb\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.403444 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.403355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1407c308-c2b5-4da0-b79a-78ffd4efaf17-maas-api-tls\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.403990 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.403938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1407c308-c2b5-4da0-b79a-78ffd4efaf17-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.405815 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.405794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1407c308-c2b5-4da0-b79a-78ffd4efaf17-maas-api-tls\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.410278 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.410254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:03.412241 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.412218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79cnb\" (UniqueName: \"kubernetes.io/projected/1407c308-c2b5-4da0-b79a-78ffd4efaf17-kube-api-access-79cnb\") pod \"maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.468963 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.468938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:12:03.541125 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.541098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:03.681334 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.681308    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"]
Jul 18 14:12:03.683086 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:12:03.683055    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1407c308_c2b5_4da0_b79a_78ffd4efaf17.slice/crio-70be2ef6ab47ad4a3cc937339fbfb45d1f5826a61aca63e7331215192ab12106 WatchSource:0}: Error finding container 70be2ef6ab47ad4a3cc937339fbfb45d1f5826a61aca63e7331215192ab12106: Status 404 returned error can't find the container with id 70be2ef6ab47ad4a3cc937339fbfb45d1f5826a61aca63e7331215192ab12106
Jul 18 14:12:03.819632 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.819601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:03.920144 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:03.920116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:04.503566 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:04.503519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx" event={"ID":"1407c308-c2b5-4da0-b79a-78ffd4efaf17","Type":"ContainerStarted","Data":"9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0"}
Jul 18 14:12:04.504098 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:04.503592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx" event={"ID":"1407c308-c2b5-4da0-b79a-78ffd4efaf17","Type":"ContainerStarted","Data":"70be2ef6ab47ad4a3cc937339fbfb45d1f5826a61aca63e7331215192ab12106"}
Jul 18 14:12:04.504478 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:04.504449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:04.521388 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:04.521327    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx" podStartSLOduration=1.192845177 podStartE2EDuration="1.521310446s" totalImagesPullingTime="328.465269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:12:04.520022908 +0000 UTC m=+2374.176713038" watchObservedRunningTime="2026-07-18 14:12:04.521310446 +0000 UTC m=+2374.178000573"
Jul 18 14:12:07.127180 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:07.127151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:07.609720 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:07.609682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:10.481205 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:10.481166    2577 patch_prober.go:28] interesting pod/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 18 14:12:10.481601 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:10.481230    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 14:12:11.521935 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.521911    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:11.540686 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.540661    2577 generic.go:356] "Generic (PLEG): container finished" podID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerID="c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c" exitCode=0
Jul 18 14:12:11.540811 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.540737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" event={"ID":"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4","Type":"ContainerDied","Data":"c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"}
Jul 18 14:12:11.541174 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.541158    2577 scope.go:117] "RemoveContainer" containerID="c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"
Jul 18 14:12:11.542105 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.542077    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:12:11.542168 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.542142    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:12:11.542207 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:11.542172    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:12:12.547555 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:12.547522    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" event={"ID":"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4","Type":"ContainerStarted","Data":"754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a"}
Jul 18 14:12:13.336543 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:13.336508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:12:13.336735 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:13.336685    2577 patch_prober.go:28] interesting pod/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 18 14:12:13.336735 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:13.336726    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 18 14:12:13.929260 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:13.929222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:14.336695 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:14.336661    2577 patch_prober.go:28] interesting pod/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 18 14:12:14.336871 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:14.336716    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 18 14:12:15.336914 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:15.336879    2577 patch_prober.go:28] interesting pod/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 18 14:12:15.337331 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:15.336961    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 18 14:12:16.148574 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:16.148541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:16.249523 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:16.249488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:16.336424 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:16.336388    2577 patch_prober.go:28] interesting pod/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 18 14:12:16.336624 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:16.336461    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 18 14:12:17.340377 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:17.340342    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:12:17.340806 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:17.340607    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:12:17.341239 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:17.341222    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:12:24.506458 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:24.506425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:24.608445 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:24.608417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:26.137973 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:26.137941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:26.239033 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:26.239005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:30.894029 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:30.893995    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:12:30.901110 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:30.901088    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:12:30.902994 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:30.902977    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:12:30.909258 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:30.909238    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:12:31.358670 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.358635    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"]
Jul 18 14:12:31.358918 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.358872    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" podUID="a017a9e9-183b-48da-805c-ecfa13b9de40" containerName="maas-api" containerID="cri-o://bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd" gracePeriod=30
Jul 18 14:12:31.380670 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.380648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:31.605277 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.605257    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:12:31.638081 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.638007    2577 generic.go:356] "Generic (PLEG): container finished" podID="a017a9e9-183b-48da-805c-ecfa13b9de40" containerID="bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd" exitCode=0
Jul 18 14:12:31.638081 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.638055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" event={"ID":"a017a9e9-183b-48da-805c-ecfa13b9de40","Type":"ContainerDied","Data":"bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd"}
Jul 18 14:12:31.638081 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.638065    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"
Jul 18 14:12:31.638299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.638096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6" event={"ID":"a017a9e9-183b-48da-805c-ecfa13b9de40","Type":"ContainerDied","Data":"a0816eb54c9849e015b6d75505a1ee2a551c204788f088672d476055f140e05a"}
Jul 18 14:12:31.638299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.638131    2577 scope.go:117] "RemoveContainer" containerID="bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd"
Jul 18 14:12:31.648718 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.648698    2577 scope.go:117] "RemoveContainer" containerID="bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd"
Jul 18 14:12:31.649012 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:12:31.648987    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd\": container with ID starting with bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd not found: ID does not exist" containerID="bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd"
Jul 18 14:12:31.649107 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.649025    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd"} err="failed to get container status \"bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd\": rpc error: code = NotFound desc = could not find container \"bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd\": container with ID starting with bc1e21612a97d44154673d42af831a66b2804727dec34c311024c6a87b4badcd not found: ID does not exist"
Jul 18 14:12:31.757476 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.757453    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls\") pod \"a017a9e9-183b-48da-805c-ecfa13b9de40\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") "
Jul 18 14:12:31.757581 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.757481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a017a9e9-183b-48da-805c-ecfa13b9de40-openshift-service-ca\") pod \"a017a9e9-183b-48da-805c-ecfa13b9de40\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") "
Jul 18 14:12:31.757581 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.757500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cf7d\" (UniqueName: \"kubernetes.io/projected/a017a9e9-183b-48da-805c-ecfa13b9de40-kube-api-access-6cf7d\") pod \"a017a9e9-183b-48da-805c-ecfa13b9de40\" (UID: \"a017a9e9-183b-48da-805c-ecfa13b9de40\") "
Jul 18 14:12:31.757936 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.757915    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a017a9e9-183b-48da-805c-ecfa13b9de40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a017a9e9-183b-48da-805c-ecfa13b9de40" (UID: "a017a9e9-183b-48da-805c-ecfa13b9de40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:12:31.759429 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.759409    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a017a9e9-183b-48da-805c-ecfa13b9de40" (UID: "a017a9e9-183b-48da-805c-ecfa13b9de40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:12:31.759551 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.759532    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a017a9e9-183b-48da-805c-ecfa13b9de40-kube-api-access-6cf7d" (OuterVolumeSpecName: "kube-api-access-6cf7d") pod "a017a9e9-183b-48da-805c-ecfa13b9de40" (UID: "a017a9e9-183b-48da-805c-ecfa13b9de40"). InnerVolumeSpecName "kube-api-access-6cf7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:12:31.858622 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.858597    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a017a9e9-183b-48da-805c-ecfa13b9de40-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:31.858622 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.858621    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a017a9e9-183b-48da-805c-ecfa13b9de40-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:31.858754 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.858630    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cf7d\" (UniqueName: \"kubernetes.io/projected/a017a9e9-183b-48da-805c-ecfa13b9de40-kube-api-access-6cf7d\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:31.959503 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.959476    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"]
Jul 18 14:12:31.962719 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:31.962699    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31be2e-b4876c7b4-gj8b6"]
Jul 18 14:12:32.801955 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:32.801926    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a017a9e9-183b-48da-805c-ecfa13b9de40" path="/var/lib/kubelet/pods/a017a9e9-183b-48da-805c-ecfa13b9de40/volumes"
Jul 18 14:12:35.551211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:35.551180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:35.926973 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:35.926904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:41.743327 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:41.743273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:41.801351 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:41.801317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:41.978977 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:41.978941    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"]
Jul 18 14:12:41.979228 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:41.979197    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy" containerID="cri-o://754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a" gracePeriod=30
Jul 18 14:12:42.015727 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:42.015582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:42.213299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:42.213265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:42.830357 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:42.830322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:42.929164 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:42.929121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:47.019867 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:12:47.019832    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f12abdd_8c92_4c5d_ab86_1e77e1a203a4.slice/crio-conmon-754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a.scope\": RecentStats: unable to find data in memory cache]"
Jul 18 14:12:47.233959 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.233933    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:12:47.292809 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292780    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istiod-ca-cert\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-envoy\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-credential-socket\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xllhd\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-kube-api-access-xllhd\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-token\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-data\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292958    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-podinfo\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.292973    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-socket\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293397 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293084    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-certs\") pod \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\" (UID: \"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4\") "
Jul 18 14:12:47.293397 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293236    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:12:47.293397 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293338    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:12:47.293397 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293352    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:12:47.293583 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293544    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:12:47.293699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293681    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.293756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293706    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istiod-ca-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.293756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293719    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-credential-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.293756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.293733    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-workload-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.294130 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.294101    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:12:47.295329 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.295297    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-kube-api-access-xllhd" (OuterVolumeSpecName: "kube-api-access-xllhd") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "kube-api-access-xllhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:12:47.295329 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.295315    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 14:12:47.295567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.295549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:12:47.295567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.295554    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" (UID: "0f12abdd-8c92-4c5d-ab86-1e77e1a203a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:12:47.394204 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.394171    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xllhd\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-kube-api-access-xllhd\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.394204 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.394200    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.394204 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.394210    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-data\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.394432 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.394217    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-podinfo\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.394432 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.394227    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4-istio-envoy\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:47.714838 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.714744    2577 generic.go:356] "Generic (PLEG): container finished" podID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerID="754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a" exitCode=0
Jul 18 14:12:47.714838 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.714798    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"
Jul 18 14:12:47.714838 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.714810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" event={"ID":"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4","Type":"ContainerDied","Data":"754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a"}
Jul 18 14:12:47.715161 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.714855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv" event={"ID":"0f12abdd-8c92-4c5d-ab86-1e77e1a203a4","Type":"ContainerDied","Data":"0f490c2a646350dfaa6be66431ceaa25f95a5af9302a1488543b871a2aa5d20c"}
Jul 18 14:12:47.715161 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.714880    2577 scope.go:117] "RemoveContainer" containerID="754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a"
Jul 18 14:12:47.724268 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.724245    2577 scope.go:117] "RemoveContainer" containerID="c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"
Jul 18 14:12:47.732001 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.731983    2577 scope.go:117] "RemoveContainer" containerID="754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a"
Jul 18 14:12:47.732274 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:12:47.732252    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a\": container with ID starting with 754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a not found: ID does not exist" containerID="754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a"
Jul 18 14:12:47.732354 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.732284    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a"} err="failed to get container status \"754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a\": rpc error: code = NotFound desc = could not find container \"754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a\": container with ID starting with 754af5e6b2d6cfde291925e0b1bdb3b9a81ad427f437bbdf174116c067c9749a not found: ID does not exist"
Jul 18 14:12:47.732354 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.732307    2577 scope.go:117] "RemoveContainer" containerID="c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"
Jul 18 14:12:47.732537 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:12:47.732519    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c\": container with ID starting with c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c not found: ID does not exist" containerID="c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"
Jul 18 14:12:47.732584 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.732544    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c"} err="failed to get container status \"c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c\": rpc error: code = NotFound desc = could not find container \"c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c\": container with ID starting with c72856f5c3f33e3bb197fb4ae3ba52bd7f8e4d5285a53626540b7b03cd381b5c not found: ID does not exist"
Jul 18 14:12:47.735990 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.735970    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"]
Jul 18 14:12:47.740156 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:47.740134    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-31be2e-openshift-default-7b5dffc9-rpwgv"]
Jul 18 14:12:48.801590 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:48.801560    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" path="/var/lib/kubelet/pods/0f12abdd-8c92-4c5d-ab86-1e77e1a203a4/volumes"
Jul 18 14:12:57.835473 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:57.835387    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"]
Jul 18 14:12:57.836026 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:57.835720    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx" podUID="1407c308-c2b5-4da0-b79a-78ffd4efaf17" containerName="maas-api" containerID="cri-o://9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0" gracePeriod=30
Jul 18 14:12:57.862962 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:57.862935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:12:58.094306 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.094241    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:58.192523 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.192483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1407c308-c2b5-4da0-b79a-78ffd4efaf17-maas-api-tls\") pod \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") "
Jul 18 14:12:58.192724 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.192581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79cnb\" (UniqueName: \"kubernetes.io/projected/1407c308-c2b5-4da0-b79a-78ffd4efaf17-kube-api-access-79cnb\") pod \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") "
Jul 18 14:12:58.192724 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.192653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1407c308-c2b5-4da0-b79a-78ffd4efaf17-openshift-service-ca\") pod \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\" (UID: \"1407c308-c2b5-4da0-b79a-78ffd4efaf17\") "
Jul 18 14:12:58.192971 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.192939    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1407c308-c2b5-4da0-b79a-78ffd4efaf17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1407c308-c2b5-4da0-b79a-78ffd4efaf17" (UID: "1407c308-c2b5-4da0-b79a-78ffd4efaf17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:12:58.194685 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.194657    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1407c308-c2b5-4da0-b79a-78ffd4efaf17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1407c308-c2b5-4da0-b79a-78ffd4efaf17" (UID: "1407c308-c2b5-4da0-b79a-78ffd4efaf17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:12:58.194781 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.194719    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1407c308-c2b5-4da0-b79a-78ffd4efaf17-kube-api-access-79cnb" (OuterVolumeSpecName: "kube-api-access-79cnb") pod "1407c308-c2b5-4da0-b79a-78ffd4efaf17" (UID: "1407c308-c2b5-4da0-b79a-78ffd4efaf17"). InnerVolumeSpecName "kube-api-access-79cnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:12:58.293525 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.293480    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1407c308-c2b5-4da0-b79a-78ffd4efaf17-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:58.293525 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.293521    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79cnb\" (UniqueName: \"kubernetes.io/projected/1407c308-c2b5-4da0-b79a-78ffd4efaf17-kube-api-access-79cnb\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:58.293797 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.293536    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1407c308-c2b5-4da0-b79a-78ffd4efaf17-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:12:58.771123 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.771076    2577 generic.go:356] "Generic (PLEG): container finished" podID="1407c308-c2b5-4da0-b79a-78ffd4efaf17" containerID="9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0" exitCode=0
Jul 18 14:12:58.771123 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.771124    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"
Jul 18 14:12:58.771382 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.771125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx" event={"ID":"1407c308-c2b5-4da0-b79a-78ffd4efaf17","Type":"ContainerDied","Data":"9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0"}
Jul 18 14:12:58.771382 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.771252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx" event={"ID":"1407c308-c2b5-4da0-b79a-78ffd4efaf17","Type":"ContainerDied","Data":"70be2ef6ab47ad4a3cc937339fbfb45d1f5826a61aca63e7331215192ab12106"}
Jul 18 14:12:58.771382 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.771280    2577 scope.go:117] "RemoveContainer" containerID="9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0"
Jul 18 14:12:58.781242 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.781220    2577 scope.go:117] "RemoveContainer" containerID="9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0"
Jul 18 14:12:58.781494 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:12:58.781476    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0\": container with ID starting with 9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0 not found: ID does not exist" containerID="9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0"
Jul 18 14:12:58.781550 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.781502    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0"} err="failed to get container status \"9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0\": rpc error: code = NotFound desc = could not find container \"9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0\": container with ID starting with 9626bcc8b9bc1e72e719e5deebded37adef1af3d64f427acd779985e7866d1b0 not found: ID does not exist"
Jul 18 14:12:58.793317 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.793293    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"]
Jul 18 14:12:58.801962 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:12:58.801940    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ee9e9-84f8fdc5cd-q69tx"]
Jul 18 14:13:00.801381 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:00.801345    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1407c308-c2b5-4da0-b79a-78ffd4efaf17" path="/var/lib/kubelet/pods/1407c308-c2b5-4da0-b79a-78ffd4efaf17/volumes"
Jul 18 14:13:02.397446 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:02.397415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:02.769025 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:02.768947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:08.361413 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:08.361376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:08.417164 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:08.417132    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:08.606450 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:08.606421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:08.825269 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:08.825242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:09.139008 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.138926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:09.414190 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414101    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"]
Jul 18 14:13:09.414595 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414579    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy"
Jul 18 14:13:09.414679 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414596    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy"
Jul 18 14:13:09.414679 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414619    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1407c308-c2b5-4da0-b79a-78ffd4efaf17" containerName="maas-api"
Jul 18 14:13:09.414679 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414627    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1407c308-c2b5-4da0-b79a-78ffd4efaf17" containerName="maas-api"
Jul 18 14:13:09.414679 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a017a9e9-183b-48da-805c-ecfa13b9de40" containerName="maas-api"
Jul 18 14:13:09.414679 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414665    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a017a9e9-183b-48da-805c-ecfa13b9de40" containerName="maas-api"
Jul 18 14:13:09.414902 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414740    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1407c308-c2b5-4da0-b79a-78ffd4efaf17" containerName="maas-api"
Jul 18 14:13:09.414902 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414751    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy"
Jul 18 14:13:09.414902 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414761    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy"
Jul 18 14:13:09.414902 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.414771    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a017a9e9-183b-48da-805c-ecfa13b9de40" containerName="maas-api"
Jul 18 14:13:09.417978 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.417957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.420200 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.420180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1bdd6b-openshift-default-dockercfg-fvvsm\""
Jul 18 14:13:09.428017 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.427993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"]
Jul 18 14:13:09.436225 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.436205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:09.493179 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-socket\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493358 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1316ab10-677a-43c8-bbcf-1558ba64eafa-istiod-ca-cert\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493358 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-envoy\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493358 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwz6b\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-kube-api-access-fwz6b\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493557 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-certs\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493557 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-token\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493557 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-podinfo\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493557 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-credential-socket\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.493557 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.493516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-data\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.535367 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.535337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:09.594842 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.594801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-certs\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.594862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-token\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.594892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-podinfo\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595015 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.594924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-credential-socket\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595188 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-data\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595243 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-socket\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1316ab10-677a-43c8-bbcf-1558ba64eafa-istiod-ca-cert\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-certs\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595299 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-envoy\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595702 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-credential-socket\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595702 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwz6b\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-kube-api-access-fwz6b\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595702 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-data\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595702 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-socket\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.595979 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.595957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1316ab10-677a-43c8-bbcf-1558ba64eafa-istiod-ca-cert\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.597355 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.597335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-envoy\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.597739 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.597721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-podinfo\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.602601 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.602573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-token\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.602691 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.602645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwz6b\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-kube-api-access-fwz6b\") pod \"e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") " pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.729955 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.729870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:09.873975 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.869829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"]
Jul 18 14:13:09.873975 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:13:09.870779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1316ab10_677a_43c8_bbcf_1558ba64eafa.slice/crio-804399dea1cbc6c57274822a140858fbbe86857e090107fa34a3710d3b73e866 WatchSource:0}: Error finding container 804399dea1cbc6c57274822a140858fbbe86857e090107fa34a3710d3b73e866: Status 404 returned error can't find the container with id 804399dea1cbc6c57274822a140858fbbe86857e090107fa34a3710d3b73e866
Jul 18 14:13:09.875289 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.875240    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:13:09.875396 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.875373    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:13:09.875481 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:09.875458    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:13:10.829098 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:10.829059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" event={"ID":"1316ab10-677a-43c8-bbcf-1558ba64eafa","Type":"ContainerStarted","Data":"61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"}
Jul 18 14:13:10.829546 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:10.829119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" event={"ID":"1316ab10-677a-43c8-bbcf-1558ba64eafa","Type":"ContainerStarted","Data":"804399dea1cbc6c57274822a140858fbbe86857e090107fa34a3710d3b73e866"}
Jul 18 14:13:10.848843 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:10.848796    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podStartSLOduration=1.848781166 podStartE2EDuration="1.848781166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:13:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:13:10.8478168 +0000 UTC m=+2440.504506930" watchObservedRunningTime="2026-07-18 14:13:10.848781166 +0000 UTC m=+2440.505471295"
Jul 18 14:13:11.730195 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:11.730155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:11.735555 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:11.735528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:11.833699 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:11.833670    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:11.834607 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:11.834588    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:12.723823 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.723784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:12.856744 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.856708    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"]
Jul 18 14:13:12.857231 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.857211    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy"
Jul 18 14:13:12.857307 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.857234    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f12abdd-8c92-4c5d-ab86-1e77e1a203a4" containerName="istio-proxy"
Jul 18 14:13:12.860353 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.860332    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:12.863135 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.863117    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-1bdd6b-serving-cert\""
Jul 18 14:13:12.863889 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.863873    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-97gtb\""
Jul 18 14:13:12.866807 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.866787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:12.887416 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.887391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"]
Jul 18 14:13:12.953537 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:12.953505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:13.028317 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.028287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxfn\" (UniqueName: \"kubernetes.io/projected/16c8a01d-19e4-425c-a0f2-7a1099f27799-kube-api-access-jvxfn\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.028469 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.028333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16c8a01d-19e4-425c-a0f2-7a1099f27799-maas-api-tls\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.028469 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.028352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16c8a01d-19e4-425c-a0f2-7a1099f27799-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.096932 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.096889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:13.129250 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.129216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxfn\" (UniqueName: \"kubernetes.io/projected/16c8a01d-19e4-425c-a0f2-7a1099f27799-kube-api-access-jvxfn\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.129414 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.129268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16c8a01d-19e4-425c-a0f2-7a1099f27799-maas-api-tls\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.129414 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.129306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16c8a01d-19e4-425c-a0f2-7a1099f27799-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.130126 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.130103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16c8a01d-19e4-425c-a0f2-7a1099f27799-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.131909 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.131887    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16c8a01d-19e4-425c-a0f2-7a1099f27799-maas-api-tls\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.136155 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.136136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxfn\" (UniqueName: \"kubernetes.io/projected/16c8a01d-19e4-425c-a0f2-7a1099f27799-kube-api-access-jvxfn\") pod \"maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.171179 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.171156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:13.303018 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.302994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"]
Jul 18 14:13:13.304427 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:13:13.304387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c8a01d_19e4_425c_a0f2_7a1099f27799.slice/crio-da19c291fba3bdd8b846bb901f9cdaa9ecae98b410fbd0753386c5c664abbf17 WatchSource:0}: Error finding container da19c291fba3bdd8b846bb901f9cdaa9ecae98b410fbd0753386c5c664abbf17: Status 404 returned error can't find the container with id da19c291fba3bdd8b846bb901f9cdaa9ecae98b410fbd0753386c5c664abbf17
Jul 18 14:13:13.306106 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.306083    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 14:13:13.477847 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.477818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:13.579809 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.579722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:13.845387 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:13.845345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8" event={"ID":"16c8a01d-19e4-425c-a0f2-7a1099f27799","Type":"ContainerStarted","Data":"da19c291fba3bdd8b846bb901f9cdaa9ecae98b410fbd0753386c5c664abbf17"}
Jul 18 14:13:14.851691 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:14.851658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8" event={"ID":"16c8a01d-19e4-425c-a0f2-7a1099f27799","Type":"ContainerStarted","Data":"3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97"}
Jul 18 14:13:14.852125 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:14.851754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:14.868059 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:14.867988    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8" podStartSLOduration=2.365955697 podStartE2EDuration="2.867972933s" totalImagesPullingTime="502.017236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:13:14.866166848 +0000 UTC m=+2444.522856976" watchObservedRunningTime="2026-07-18 14:13:14.867972933 +0000 UTC m=+2444.524663092"
Jul 18 14:13:17.196425 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:17.196388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:17.514509 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:17.514409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:17.817776 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:17.817747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:18.466334 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:18.466298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:20.598220 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.598181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:20.686500 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.678819    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"]
Jul 18 14:13:20.689200 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.689161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.690866 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.690836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:20.692786 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.692752    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-66d5d1-serving-cert\""
Jul 18 14:13:20.695149 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.695124    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"]
Jul 18 14:13:20.700191 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.700066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkb7\" (UniqueName: \"kubernetes.io/projected/82eb71fb-d431-4204-8874-acf5ade97f4c-kube-api-access-9vkb7\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.700276 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.700213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82eb71fb-d431-4204-8874-acf5ade97f4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.700350 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.700285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.754400 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.754372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:20.801779 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.801745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82eb71fb-d431-4204-8874-acf5ade97f4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.801977 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.801858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.801977 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.801956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkb7\" (UniqueName: \"kubernetes.io/projected/82eb71fb-d431-4204-8874-acf5ade97f4c-kube-api-access-9vkb7\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.802271 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:13:20.802251    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-serving-cert: secret "maas-api-e2e-shared-b-66d5d1-serving-cert" not found
Jul 18 14:13:20.802417 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:13:20.802400    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls podName:82eb71fb-d431-4204-8874-acf5ade97f4c nodeName:}" failed. No retries permitted until 2026-07-18 14:13:21.302378626 +0000 UTC m=+2450.959068749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls") pod "maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" (UID: "82eb71fb-d431-4204-8874-acf5ade97f4c") : secret "maas-api-e2e-shared-b-66d5d1-serving-cert" not found
Jul 18 14:13:20.802678 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.802655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82eb71fb-d431-4204-8874-acf5ade97f4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.810853 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.810826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkb7\" (UniqueName: \"kubernetes.io/projected/82eb71fb-d431-4204-8874-acf5ade97f4c-kube-api-access-9vkb7\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:20.861631 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.861565    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:13:20.905619 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:20.905589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:21.251718 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:21.251631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:21.307770 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:21.307738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:21.310226 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:21.310203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:21.353954 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:21.351680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:21.603631 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:21.603588    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:21.949795 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:21.949725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"]
Jul 18 14:13:21.952140 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:13:21.952111    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82eb71fb_d431_4204_8874_acf5ade97f4c.slice/crio-faa91ca5432134c9c5099a0c4f1e7e5eaf627a9609a1fd57f0c5822c5be03818 WatchSource:0}: Error finding container faa91ca5432134c9c5099a0c4f1e7e5eaf627a9609a1fd57f0c5822c5be03818: Status 404 returned error can't find the container with id faa91ca5432134c9c5099a0c4f1e7e5eaf627a9609a1fd57f0c5822c5be03818
Jul 18 14:13:22.897342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:22.897000    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" event={"ID":"82eb71fb-d431-4204-8874-acf5ade97f4c","Type":"ContainerStarted","Data":"45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37"}
Jul 18 14:13:22.897342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:22.897086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" event={"ID":"82eb71fb-d431-4204-8874-acf5ade97f4c","Type":"ContainerStarted","Data":"faa91ca5432134c9c5099a0c4f1e7e5eaf627a9609a1fd57f0c5822c5be03818"}
Jul 18 14:13:22.897342 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:22.897148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:22.914675 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:22.914620    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" podStartSLOduration=2.596791842 podStartE2EDuration="2.914599673s" totalImagesPullingTime="317.807831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 14:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:13:22.91245428 +0000 UTC m=+2452.569144411" watchObservedRunningTime="2026-07-18 14:13:22.914599673 +0000 UTC m=+2452.571289802"
Jul 18 14:13:25.089468 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:25.089428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:27.876215 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:27.876170    2577 patch_prober.go:28] interesting pod/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 18 14:13:27.876708 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:27.876246    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 14:13:28.911943 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.911872    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:13:28.935738 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.935697    2577 generic.go:356] "Generic (PLEG): container finished" podID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerID="61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f" exitCode=0
Jul 18 14:13:28.935898 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.935746    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" event={"ID":"1316ab10-677a-43c8-bbcf-1558ba64eafa","Type":"ContainerDied","Data":"61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"}
Jul 18 14:13:28.936197 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.936180    2577 scope.go:117] "RemoveContainer" containerID="61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"
Jul 18 14:13:28.937618 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.937585    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:13:28.937739 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.937670    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:13:28.937739 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:28.937720    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 18 14:13:29.944914 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:29.944490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" event={"ID":"1316ab10-677a-43c8-bbcf-1558ba64eafa","Type":"ContainerStarted","Data":"5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1"}
Jul 18 14:13:30.730031 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:30.729996    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:30.730220 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:30.730191    2577 patch_prober.go:28] interesting pod/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 18 14:13:30.730274 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:30.730248    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 18 14:13:31.041070 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:31.041029    2577 scope.go:117] "RemoveContainer" containerID="12ff6132828484961fe1299c99d4a3413a40b041f751c4880e4755d998064501"
Jul 18 14:13:31.730890 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:31.730844    2577 patch_prober.go:28] interesting pod/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 18 14:13:31.731121 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:31.730904    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 18 14:13:32.730991 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:32.730954    2577 patch_prober.go:28] interesting pod/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 18 14:13:32.731413 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:32.731008    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 18 14:13:33.730847 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:33.730811    2577 patch_prober.go:28] interesting pod/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 18 14:13:33.730996 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:33.730872    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 18 14:13:34.221396 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:34.221362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:34.322661 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:34.322634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:34.734128 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:34.734098    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:34.734374 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:34.734356    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:34.735104 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:34.735084    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:13:41.964989 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:41.964951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:42.063197 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:42.063155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:13:49.799449 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:13:49.799406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:14:01.272148 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:14:01.272116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:14:23.878151 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:14:23.878077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:14:53.451189 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:14:53.451151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:14:53.548318 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:14:53.548289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:15:05.649130 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:15:05.649093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:15:23.163227 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:15:23.163194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:16:23.617239 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:16:23.617164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:16:24.623129 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:16:24.623092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:16:34.511763 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:16:34.511729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:17:08.015733 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:08.015701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:17:30.922609 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:30.922578    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:17:30.929747 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:30.929722    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:17:30.932770 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:30.932751    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:17:30.938905 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:30.938887    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrzc_3b02ccd9-1bb3-44fc-861a-42805558947a/ovn-acl-logging/0.log"
Jul 18 14:17:44.940771 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:44.940734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:17:45.921203 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:45.921169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:17:56.614383 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:17:56.614352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:18:30.039279 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:18:30.039248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:18:34.932783 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:18:34.932746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:18:35.033402 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:18:35.033371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:18:42.734903 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:18:42.734870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:18:42.836744 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:18:42.836709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:18:49.368001 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:18:49.367945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:18.764017 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:18.763921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:19.608616 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:19.608584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:38.936955 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:38.936921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:39.035976 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:39.035939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:48.716216 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:48.716184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"]
Jul 18 14:19:48.716706 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:48.716387    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8" podUID="16c8a01d-19e4-425c-a0f2-7a1099f27799" containerName="maas-api" containerID="cri-o://3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97" gracePeriod=30
Jul 18 14:19:48.736294 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:48.736261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:48.974014 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:48.973956    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:19:49.150752 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.150721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvxfn\" (UniqueName: \"kubernetes.io/projected/16c8a01d-19e4-425c-a0f2-7a1099f27799-kube-api-access-jvxfn\") pod \"16c8a01d-19e4-425c-a0f2-7a1099f27799\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") "
Jul 18 14:19:49.150752 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.150754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16c8a01d-19e4-425c-a0f2-7a1099f27799-maas-api-tls\") pod \"16c8a01d-19e4-425c-a0f2-7a1099f27799\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") "
Jul 18 14:19:49.150980 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.150799    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16c8a01d-19e4-425c-a0f2-7a1099f27799-openshift-service-ca\") pod \"16c8a01d-19e4-425c-a0f2-7a1099f27799\" (UID: \"16c8a01d-19e4-425c-a0f2-7a1099f27799\") "
Jul 18 14:19:49.151277 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.151246    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16c8a01d-19e4-425c-a0f2-7a1099f27799-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16c8a01d-19e4-425c-a0f2-7a1099f27799" (UID: "16c8a01d-19e4-425c-a0f2-7a1099f27799"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:19:49.152797 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.152769    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16c8a01d-19e4-425c-a0f2-7a1099f27799-kube-api-access-jvxfn" (OuterVolumeSpecName: "kube-api-access-jvxfn") pod "16c8a01d-19e4-425c-a0f2-7a1099f27799" (UID: "16c8a01d-19e4-425c-a0f2-7a1099f27799"). InnerVolumeSpecName "kube-api-access-jvxfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:19:49.152905 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.152839    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16c8a01d-19e4-425c-a0f2-7a1099f27799-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16c8a01d-19e4-425c-a0f2-7a1099f27799" (UID: "16c8a01d-19e4-425c-a0f2-7a1099f27799"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:19:49.251751 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.251686    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvxfn\" (UniqueName: \"kubernetes.io/projected/16c8a01d-19e4-425c-a0f2-7a1099f27799-kube-api-access-jvxfn\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:19:49.251751 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.251716    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16c8a01d-19e4-425c-a0f2-7a1099f27799-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:19:49.251751 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.251725    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16c8a01d-19e4-425c-a0f2-7a1099f27799-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:19:49.743366 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.743328    2577 generic.go:356] "Generic (PLEG): container finished" podID="16c8a01d-19e4-425c-a0f2-7a1099f27799" containerID="3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97" exitCode=0
Jul 18 14:19:49.743736 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.743374    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"
Jul 18 14:19:49.743736 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.743415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8" event={"ID":"16c8a01d-19e4-425c-a0f2-7a1099f27799","Type":"ContainerDied","Data":"3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97"}
Jul 18 14:19:49.743736 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.743472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8" event={"ID":"16c8a01d-19e4-425c-a0f2-7a1099f27799","Type":"ContainerDied","Data":"da19c291fba3bdd8b846bb901f9cdaa9ecae98b410fbd0753386c5c664abbf17"}
Jul 18 14:19:49.743736 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.743495    2577 scope.go:117] "RemoveContainer" containerID="3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97"
Jul 18 14:19:49.752457 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.752440    2577 scope.go:117] "RemoveContainer" containerID="3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97"
Jul 18 14:19:49.752700 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:19:49.752681    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97\": container with ID starting with 3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97 not found: ID does not exist" containerID="3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97"
Jul 18 14:19:49.752746 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.752710    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97"} err="failed to get container status \"3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97\": rpc error: code = NotFound desc = could not find container \"3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97\": container with ID starting with 3afdd8cfeea0e9cd7db0035a7a6a1423938c17d406d2e472820deb87d0578f97 not found: ID does not exist"
Jul 18 14:19:49.763444 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.763421    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"]
Jul 18 14:19:49.767135 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:49.767113    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1bdd6b-c8b779586-jlvc8"]
Jul 18 14:19:50.801423 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:50.801392    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16c8a01d-19e4-425c-a0f2-7a1099f27799" path="/var/lib/kubelet/pods/16c8a01d-19e4-425c-a0f2-7a1099f27799/volumes"
Jul 18 14:19:53.419964 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:53.419927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:53.800970 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:53.800938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:54.242151 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:54.242066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:54.345609 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:54.345572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:59.798121 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:59.798083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:19:59.852863 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:19:59.852823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:00.036234 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:00.036197    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"]
Jul 18 14:20:00.036508 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:00.036483    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" containerID="cri-o://5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1" gracePeriod=30
Jul 18 14:20:00.050911 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:00.050824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:00.274980 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:00.274943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:00.964060 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:00.964018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:01.064724 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:01.064675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:04.737773 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:04.737729    2577 patch_prober.go:28] interesting pod/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 14:20:04.738181 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:04.737804    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 14:20:05.293370 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.293335    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:20:05.379962 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.379876    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-certs\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.379962 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.379956    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-socket\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.379983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwz6b\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-kube-api-access-fwz6b\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-podinfo\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380073    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-credential-socket\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-data\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380138    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1316ab10-677a-43c8-bbcf-1558ba64eafa-istiod-ca-cert\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-envoy\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380211 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380209    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-token\") pod \"1316ab10-677a-43c8-bbcf-1558ba64eafa\" (UID: \"1316ab10-677a-43c8-bbcf-1558ba64eafa\") "
Jul 18 14:20:05.380567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380228    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:20:05.380567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380281    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:20:05.380567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380520    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-certs\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.380567 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380542    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-workload-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.380782 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380717    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1316ab10-677a-43c8-bbcf-1558ba64eafa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:20:05.380833 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.380789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:20:05.381124 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.381101    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-data" (OuterVolumeSpecName: "istio-data") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:20:05.382926 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.382895    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-kube-api-access-fwz6b" (OuterVolumeSpecName: "kube-api-access-fwz6b") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "kube-api-access-fwz6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:20:05.383021 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.382925    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 14:20:05.383021 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.382938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-token" (OuterVolumeSpecName: "istio-token") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:20:05.383021 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.382896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1316ab10-677a-43c8-bbcf-1558ba64eafa" (UID: "1316ab10-677a-43c8-bbcf-1558ba64eafa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 14:20:05.481675 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481630    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.481675 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481670    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwz6b\" (UniqueName: \"kubernetes.io/projected/1316ab10-677a-43c8-bbcf-1558ba64eafa-kube-api-access-fwz6b\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.481675 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481684    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-podinfo\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.481935 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481697    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-credential-socket\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.481935 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481708    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-data\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.481935 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481736    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1316ab10-677a-43c8-bbcf-1558ba64eafa-istiod-ca-cert\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.481935 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.481746    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1316ab10-677a-43c8-bbcf-1558ba64eafa-istio-envoy\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:05.830895 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.830862    2577 generic.go:356] "Generic (PLEG): container finished" podID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerID="5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1" exitCode=0
Jul 18 14:20:05.831390 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.830906    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"
Jul 18 14:20:05.831390 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.830939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" event={"ID":"1316ab10-677a-43c8-bbcf-1558ba64eafa","Type":"ContainerDied","Data":"5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1"}
Jul 18 14:20:05.831390 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.831007    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f" event={"ID":"1316ab10-677a-43c8-bbcf-1558ba64eafa","Type":"ContainerDied","Data":"804399dea1cbc6c57274822a140858fbbe86857e090107fa34a3710d3b73e866"}
Jul 18 14:20:05.831390 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.831062    2577 scope.go:117] "RemoveContainer" containerID="5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1"
Jul 18 14:20:05.840322 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.840301    2577 scope.go:117] "RemoveContainer" containerID="61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"
Jul 18 14:20:05.848487 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.848467    2577 scope.go:117] "RemoveContainer" containerID="5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1"
Jul 18 14:20:05.848774 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:20:05.848753    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1\": container with ID starting with 5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1 not found: ID does not exist" containerID="5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1"
Jul 18 14:20:05.848826 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.848782    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1"} err="failed to get container status \"5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1\": rpc error: code = NotFound desc = could not find container \"5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1\": container with ID starting with 5781d36fcfe765735e7cfcf27bc6b39021b4d4ee39329cf5bc3e40db59b421c1 not found: ID does not exist"
Jul 18 14:20:05.848826 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.848801    2577 scope.go:117] "RemoveContainer" containerID="61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"
Jul 18 14:20:05.849061 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:20:05.849026    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f\": container with ID starting with 61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f not found: ID does not exist" containerID="61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"
Jul 18 14:20:05.849179 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.849068    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f"} err="failed to get container status \"61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f\": rpc error: code = NotFound desc = could not find container \"61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f\": container with ID starting with 61dcc6002863eb4d93eb56592d7f284c954a207366424e3905972df50facb65f not found: ID does not exist"
Jul 18 14:20:05.852245 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.852219    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"]
Jul 18 14:20:05.858148 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.858124    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1bdd6b-openshift-default-d6fcb7b7-2tp7f"]
Jul 18 14:20:05.893251 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.893221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:05.947318 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:05.947286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:06.802759 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:06.802719    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" path="/var/lib/kubelet/pods/1316ab10-677a-43c8-bbcf-1558ba64eafa/volumes"
Jul 18 14:20:10.919890 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:10.919857    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"]
Jul 18 14:20:10.920373 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:10.920147    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" podUID="82eb71fb-d431-4204-8874-acf5ade97f4c" containerName="maas-api" containerID="cri-o://45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37" gracePeriod=30
Jul 18 14:20:10.946006 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:10.945978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:11.173258 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.173208    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:20:11.233642 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.233612    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls\") pod \"82eb71fb-d431-4204-8874-acf5ade97f4c\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") "
Jul 18 14:20:11.233806 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.233696    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vkb7\" (UniqueName: \"kubernetes.io/projected/82eb71fb-d431-4204-8874-acf5ade97f4c-kube-api-access-9vkb7\") pod \"82eb71fb-d431-4204-8874-acf5ade97f4c\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") "
Jul 18 14:20:11.233806 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.233740    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82eb71fb-d431-4204-8874-acf5ade97f4c-openshift-service-ca\") pod \"82eb71fb-d431-4204-8874-acf5ade97f4c\" (UID: \"82eb71fb-d431-4204-8874-acf5ade97f4c\") "
Jul 18 14:20:11.234199 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.234173    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82eb71fb-d431-4204-8874-acf5ade97f4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82eb71fb-d431-4204-8874-acf5ade97f4c" (UID: "82eb71fb-d431-4204-8874-acf5ade97f4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 14:20:11.235575 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.235551    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82eb71fb-d431-4204-8874-acf5ade97f4c" (UID: "82eb71fb-d431-4204-8874-acf5ade97f4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 14:20:11.235741 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.235720    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82eb71fb-d431-4204-8874-acf5ade97f4c-kube-api-access-9vkb7" (OuterVolumeSpecName: "kube-api-access-9vkb7") pod "82eb71fb-d431-4204-8874-acf5ade97f4c" (UID: "82eb71fb-d431-4204-8874-acf5ade97f4c"). InnerVolumeSpecName "kube-api-access-9vkb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 14:20:11.334878 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.334851    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vkb7\" (UniqueName: \"kubernetes.io/projected/82eb71fb-d431-4204-8874-acf5ade97f4c-kube-api-access-9vkb7\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:11.334878 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.334876    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82eb71fb-d431-4204-8874-acf5ade97f4c-openshift-service-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:11.335016 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.334887    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82eb71fb-d431-4204-8874-acf5ade97f4c-maas-api-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\""
Jul 18 14:20:11.860653 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.860617    2577 generic.go:356] "Generic (PLEG): container finished" podID="82eb71fb-d431-4204-8874-acf5ade97f4c" containerID="45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37" exitCode=0
Jul 18 14:20:11.860860 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.860657    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"
Jul 18 14:20:11.860860 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.860697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" event={"ID":"82eb71fb-d431-4204-8874-acf5ade97f4c","Type":"ContainerDied","Data":"45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37"}
Jul 18 14:20:11.860860 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.860743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf" event={"ID":"82eb71fb-d431-4204-8874-acf5ade97f4c","Type":"ContainerDied","Data":"faa91ca5432134c9c5099a0c4f1e7e5eaf627a9609a1fd57f0c5822c5be03818"}
Jul 18 14:20:11.860860 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.860765    2577 scope.go:117] "RemoveContainer" containerID="45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37"
Jul 18 14:20:11.869766 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.869751    2577 scope.go:117] "RemoveContainer" containerID="45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37"
Jul 18 14:20:11.870008 ip-10-0-128-123 kubenswrapper[2577]: E0718 14:20:11.869991    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37\": container with ID starting with 45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37 not found: ID does not exist" containerID="45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37"
Jul 18 14:20:11.870098 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.870015    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37"} err="failed to get container status \"45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37\": rpc error: code = NotFound desc = could not find container \"45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37\": container with ID starting with 45d0d00cbb54528cdff27c588e63ff66bfd3d831e21b39b573563bd228842a37 not found: ID does not exist"
Jul 18 14:20:11.879750 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.879729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"]
Jul 18 14:20:11.883168 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:11.883150    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-66d5d1-567d5b59f8-bfsjf"]
Jul 18 14:20:12.802163 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:12.802132    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82eb71fb-d431-4204-8874-acf5ade97f4c" path="/var/lib/kubelet/pods/82eb71fb-d431-4204-8874-acf5ade97f4c/volumes"
Jul 18 14:20:15.463696 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:15.463662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:15.848294 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:15.848257    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:21.694295 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:21.694254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:21.795161 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:21.795128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:21.967533 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:21.967456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:22.176832 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:22.176794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:27.859982 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:27.859905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f744f9d59-qrv5k_77ac0aff-7a4c-4b52-8257-aa89371b6abd/manager/0.log"
Jul 18 14:20:28.310689 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:28.310661    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-mdsp9_c6575cad-578b-4d1a-a883-d27b724456f8/manager/0.log"
Jul 18 14:20:28.332780 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:28.332753    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-mdsp9_c6575cad-578b-4d1a-a883-d27b724456f8/copy-manifests/0.log"
Jul 18 14:20:28.353581 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:28.353560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:29.090187 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:29.090153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8nbxd_983920a3-8a28-428a-b439-91f3603926d9/postgres/0.log"
Jul 18 14:20:31.405424 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:31.405317    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jr5xd_55be83ab-4c95-4a71-9c08-b6a9ead5eea1/istio-proxy/0.log"
Jul 18 14:20:31.515835 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:31.515806    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-q8kj4_f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e/discovery/0.log"
Jul 18 14:20:32.169351 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:32.169321    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b6d57b78-p692k_d3d2f953-ec65-428d-94f7-222faa182970/router/0.log"
Jul 18 14:20:32.714627 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:32.714601    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hzwcq_8aca100b-507f-4428-94ce-2b294ef5862d/main/0.log"
Jul 18 14:20:32.930120 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:32.930093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-n8dcr_57ec7d97-1938-489f-b361-c834d8e6dabc/main/0.log"
Jul 18 14:20:38.397644 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:38.397607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:38.462142 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:38.462108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log" limit=1024
Jul 18 14:20:41.202664 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:41.202635    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2849v_1cb6a15d-8760-41f7-a007-d22df913e121/global-pull-secret-syncer/0.log"
Jul 18 14:20:41.324601 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:41.324575    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6dn7v_f4c6ff44-14b9-4aab-9861-110804401955/konnectivity-agent/0.log"
Jul 18 14:20:41.410484 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:41.410461    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-123.ec2.internal_8926567829fff2679ba8a397c742012f/haproxy/0.log"
Jul 18 14:20:48.861392 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:48.861361    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-5s54r_46e64f4d-3a2b-433c-82e7-1ae26cfde9dd/kube-state-metrics/0.log"
Jul 18 14:20:48.884875 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:48.884851    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-5s54r_46e64f4d-3a2b-433c-82e7-1ae26cfde9dd/kube-rbac-proxy-main/0.log"
Jul 18 14:20:48.905603 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:48.905588    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-5s54r_46e64f4d-3a2b-433c-82e7-1ae26cfde9dd/kube-rbac-proxy-self/0.log"
Jul 18 14:20:48.969450 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:48.969429    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-xvj84_09483280-46b3-4430-8f9c-deb20663f65d/monitoring-plugin/0.log"
Jul 18 14:20:49.013088 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:49.013073    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-56khk_82c3a650-589d-4ce7-82a4-c71623973731/node-exporter/0.log"
Jul 18 14:20:49.036221 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:49.036200    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-56khk_82c3a650-589d-4ce7-82a4-c71623973731/kube-rbac-proxy/0.log"
Jul 18 14:20:49.058716 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:49.058694    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-56khk_82c3a650-589d-4ce7-82a4-c71623973731/init-textfile/0.log"
Jul 18 14:20:49.471340 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:49.471300    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-bfvjq_a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df/prometheus-operator/0.log"
Jul 18 14:20:49.491550 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:49.491527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-bfvjq_a1ac7f31-4b48-44a3-a5e6-d2ba32cee1df/kube-rbac-proxy/0.log"
Jul 18 14:20:51.560149 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.560117    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/1.log"
Jul 18 14:20:51.564959 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.564934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2dpwt_6e46d182-d684-474c-869a-2d4571579100/console-operator/2.log"
Jul 18 14:20:51.832459 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832386    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"]
Jul 18 14:20:51.832729 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832716    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy"
Jul 18 14:20:51.832798 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832731    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy"
Jul 18 14:20:51.832798 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832751    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16c8a01d-19e4-425c-a0f2-7a1099f27799" containerName="maas-api"
Jul 18 14:20:51.832798 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832756    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="16c8a01d-19e4-425c-a0f2-7a1099f27799" containerName="maas-api"
Jul 18 14:20:51.832798 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832768    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82eb71fb-d431-4204-8874-acf5ade97f4c" containerName="maas-api"
Jul 18 14:20:51.832798 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832774    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="82eb71fb-d431-4204-8874-acf5ade97f4c" containerName="maas-api"
Jul 18 14:20:51.833067 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832886    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="82eb71fb-d431-4204-8874-acf5ade97f4c" containerName="maas-api"
Jul 18 14:20:51.833067 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832900    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy"
Jul 18 14:20:51.833067 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832912    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1316ab10-677a-43c8-bbcf-1558ba64eafa" containerName="istio-proxy"
Jul 18 14:20:51.833067 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.832922    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="16c8a01d-19e4-425c-a0f2-7a1099f27799" containerName="maas-api"
Jul 18 14:20:51.835737 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.835721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.838112 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.838091    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bq9sq\"/\"openshift-service-ca.crt\""
Jul 18 14:20:51.838242 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.838117    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bq9sq\"/\"kube-root-ca.crt\""
Jul 18 14:20:51.838968 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.838954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bq9sq\"/\"default-dockercfg-r9wfw\""
Jul 18 14:20:51.844349 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.844328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"]
Jul 18 14:20:51.863756 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.863732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-lib-modules\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.863880 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.863783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks4jx\" (UniqueName: \"kubernetes.io/projected/4ea610f5-858c-497c-9e3f-d341570dea3c-kube-api-access-ks4jx\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.863880 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.863839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-sys\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.863880 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.863864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-proc\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.864080 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.863942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-podres\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.964747 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-podres\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.964889 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-lib-modules\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.964889 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks4jx\" (UniqueName: \"kubernetes.io/projected/4ea610f5-858c-497c-9e3f-d341570dea3c-kube-api-access-ks4jx\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.965010 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-podres\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.965010 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-lib-modules\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.965010 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-sys\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.965010 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-proc\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.965010 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.964990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-sys\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.965237 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.965026    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4ea610f5-858c-497c-9e3f-d341570dea3c-proc\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:51.972692 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:51.972670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks4jx\" (UniqueName: \"kubernetes.io/projected/4ea610f5-858c-497c-9e3f-d341570dea3c-kube-api-access-ks4jx\") pod \"perf-node-gather-daemonset-k9k5v\" (UID: \"4ea610f5-858c-497c-9e3f-d341570dea3c\") " pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:52.146976 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:52.146915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:52.276696 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:52.276623    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"]
Jul 18 14:20:52.279168 ip-10-0-128-123 kubenswrapper[2577]: W0718 14:20:52.279137    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4ea610f5_858c_497c_9e3f_d341570dea3c.slice/crio-346840b5e15797a0bef3fa4a4da5d1a3d4ee7d83f3e88609d3dbfbe75afdf206 WatchSource:0}: Error finding container 346840b5e15797a0bef3fa4a4da5d1a3d4ee7d83f3e88609d3dbfbe75afdf206: Status 404 returned error can't find the container with id 346840b5e15797a0bef3fa4a4da5d1a3d4ee7d83f3e88609d3dbfbe75afdf206
Jul 18 14:20:52.844489 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:52.844461    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-mb2ch_6aab0b5a-dbc2-4a75-bb02-644272f6dec2/volume-data-source-validator/0.log"
Jul 18 14:20:53.065362 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.065326    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v" event={"ID":"4ea610f5-858c-497c-9e3f-d341570dea3c","Type":"ContainerStarted","Data":"05172538ee0fb7bdf2da867a515b98cda387b2a512596a73ea2b33b0403ec9b9"}
Jul 18 14:20:53.065550 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.065375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v" event={"ID":"4ea610f5-858c-497c-9e3f-d341570dea3c","Type":"ContainerStarted","Data":"346840b5e15797a0bef3fa4a4da5d1a3d4ee7d83f3e88609d3dbfbe75afdf206"}
Jul 18 14:20:53.065550 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.065402    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:53.080904 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.080860    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v" podStartSLOduration=2.080847043 podStartE2EDuration="2.080847043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 14:20:53.079833373 +0000 UTC m=+2902.736523504" watchObservedRunningTime="2026-07-18 14:20:53.080847043 +0000 UTC m=+2902.737537171"
Jul 18 14:20:53.799527 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.799497    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9j6vs_894a67ce-7ec2-4ffb-b31a-3b62e845a2f4/dns/0.log"
Jul 18 14:20:53.817940 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.817911    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9j6vs_894a67ce-7ec2-4ffb-b31a-3b62e845a2f4/kube-rbac-proxy/0.log"
Jul 18 14:20:53.937106 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:53.937078    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9fnb7_9dd4addb-d362-4a5a-96ff-4bc28f94e2f3/dns-node-resolver/0.log"
Jul 18 14:20:54.489217 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:54.489188    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4v2rl_cb9103a2-100d-4187-91fc-8708bbf5cd35/node-ca/0.log"
Jul 18 14:20:55.416873 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:55.416844    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jr5xd_55be83ab-4c95-4a71-9c08-b6a9ead5eea1/istio-proxy/0.log"
Jul 18 14:20:55.536605 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:55.536580    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-q8kj4_f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e/discovery/0.log"
Jul 18 14:20:55.710924 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:55.710857    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b6d57b78-p692k_d3d2f953-ec65-428d-94f7-222faa182970/router/0.log"
Jul 18 14:20:56.380840 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:56.380811    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7hw6q_19c69f7e-d198-4022-abaf-cb8b8f3b329b/serve-healthcheck-canary/0.log"
Jul 18 14:20:56.887079 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:56.887049    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-5nkpc_9e57eaa4-6de2-498b-830c-690e2fdce624/insights-operator/0.log"
Jul 18 14:20:56.887440 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:56.887392    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-5nkpc_9e57eaa4-6de2-498b-830c-690e2fdce624/insights-operator/1.log"
Jul 18 14:20:56.989167 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:56.989135    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2hlkh_1b4d5a23-b14b-4260-87d0-0f838919b077/kube-rbac-proxy/0.log"
Jul 18 14:20:57.007842 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:57.007818    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2hlkh_1b4d5a23-b14b-4260-87d0-0f838919b077/exporter/0.log"
Jul 18 14:20:57.025861 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:57.025842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2hlkh_1b4d5a23-b14b-4260-87d0-0f838919b077/extractor/0.log"
Jul 18 14:20:59.079192 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:59.079166    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bq9sq/perf-node-gather-daemonset-k9k5v"
Jul 18 14:20:59.600966 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:59.600942    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f744f9d59-qrv5k_77ac0aff-7a4c-4b52-8257-aa89371b6abd/manager/0.log"
Jul 18 14:20:59.825970 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:59.825937    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-mdsp9_c6575cad-578b-4d1a-a883-d27b724456f8/manager/0.log"
Jul 18 14:20:59.841010 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:20:59.840989    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7687b4d-mdsp9_c6575cad-578b-4d1a-a883-d27b724456f8/copy-manifests/0.log"
Jul 18 14:21:01.250518 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:01.250487    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8664c48d9f-2r8tw_1dcab15d-73a0-4e64-aa78-4511f3b43358/manager/0.log"
Jul 18 14:21:02.038381 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:02.038355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jr5xd_55be83ab-4c95-4a71-9c08-b6a9ead5eea1/istio-proxy/0.log"
Jul 18 14:21:02.151358 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:02.151330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-q8kj4_f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e/discovery/0.log"
Jul 18 14:21:02.277541 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:02.277499    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b6d57b78-p692k_d3d2f953-ec65-428d-94f7-222faa182970/router/0.log"
Jul 18 14:21:03.410140 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:03.410059    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jr5xd_55be83ab-4c95-4a71-9c08-b6a9ead5eea1/istio-proxy/0.log"
Jul 18 14:21:03.534990 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:03.534963    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-q8kj4_f7e1ef6e-7376-4bc3-87d5-9da4b6b9048e/discovery/0.log"
Jul 18 14:21:03.693187 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:03.693124    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b6d57b78-p692k_d3d2f953-ec65-428d-94f7-222faa182970/router/0.log"
Jul 18 14:21:06.673599 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:06.673565    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-jmc6r_b59163b6-7d4a-4c22-ae0c-98ffaefc7f35/kube-storage-version-migrator-operator/1.log"
Jul 18 14:21:06.674453 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:06.674435    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-jmc6r_b59163b6-7d4a-4c22-ae0c-98ffaefc7f35/kube-storage-version-migrator-operator/0.log"
Jul 18 14:21:07.683458 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:07.683433    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7nltq_07af34cb-f6f9-4dd5-b7b2-038a374d8201/kube-multus/0.log"
Jul 18 14:21:07.978959 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:07.978891    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/kube-multus-additional-cni-plugins/0.log"
Jul 18 14:21:07.997217 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:07.997185    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/egress-router-binary-copy/0.log"
Jul 18 14:21:08.014603 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.014588    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/cni-plugins/0.log"
Jul 18 14:21:08.033741 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.033726    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/bond-cni-plugin/0.log"
Jul 18 14:21:08.050723 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.050705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/routeoverride-cni/0.log"
Jul 18 14:21:08.069138 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.069123    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/whereabouts-cni-bincopy/0.log"
Jul 18 14:21:08.086629 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.086607    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5mqw_8bb5079e-d591-4460-9b9b-6d508caf5620/whereabouts-cni/0.log"
Jul 18 14:21:08.173931 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.173898    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5pfd2_d414579c-495e-4311-9ab8-bc2bae36afd9/network-metrics-daemon/0.log"
Jul 18 14:21:08.192001 ip-10-0-128-123 kubenswrapper[2577]: I0718 14:21:08.191984    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5pfd2_d414579c-495e-4311-9ab8-bc2bae36afd9/kube-rbac-proxy/0.log"